If I'm doing a lead over a meter, or drumming where there is some meter but I'm doing something outside that box, I should be cognizant of where 1 is anyway.
However I do unmetered music all the time, and impulsively. Where I actually have to determine barlines and beats after the fact.
You want to get to where the meter itself (eg., "4") is not something you have to think a lot about.
You can't think and play at the same time. - Sonny Rollins