Everyone is numbers. I'm going to tell you a secret. In reality, you don't need NUMBERS, you need READERS. There's a difference.
Readers can't be noticed to everyone. These are people known only to you. And, believe me, READERS are more important than NUMBERS.
Let's use me as an example.
I changed my blog last week. Name and everything. I'm not listed on Alexa Ranking. I'm not listed on Google Page rank. I have 2 GFC followers. I have 46 twitter followers and NO facebook page likes.
What do I have? I have over 1500 UNIQUE page views per day.
Now, you tell me - what's more important? The numbers or those who are actually coming by to read?
The biggest thing is to get your links out there. Join social networks so your links get around. But don't base your social reach on just your followers and connections.
Don't sweat the numbers, You'll just keep yourself from working on the things that you really should be doing it. Work on quality. And you'll get READERS.