If you pushed something (publicly) to GitHub, there's no undo: you HAVE TO change your keys now. Your keys were immediately ingested and compromised a few seconds after you pushed, and likely even before you realized what you've done.
To paraphrase from my own comment last month [1]:
"Some time ago I published my blog to GitHub, with my MailGun API key in the config file (stupid mistake, I know). In less than 12 hours, spammers had harvested the key AND sent a few thousand emails with my account, using my entire monthly limit.
Thankfully I was using the free MailGun account, which is limited to only 10,000 emails/month, so there was no material damage. And MailGun's tech support was awesome and immediately blocked the account and notified me, reseting the account after I had changed the keys."
If you're wondering how they were able to harvest GitHub commits so quickly, just read the article linked in that thread [2]. Basically you have bots drinking from GitHub's events firehose and the GHTorrent project. Every commit is monitored and harvested for passwords on the fly.
Nice, so now we need to make honeypots which post to GitHub tons of things that look like real secrets but with broken credentials, monitor logins for those things, and start a blacklist of compromised / malicious systems (which would be the things trying with those logins).
To paraphrase from my own comment last month [1]:
"Some time ago I published my blog to GitHub, with my MailGun API key in the config file (stupid mistake, I know). In less than 12 hours, spammers had harvested the key AND sent a few thousand emails with my account, using my entire monthly limit.
Thankfully I was using the free MailGun account, which is limited to only 10,000 emails/month, so there was no material damage. And MailGun's tech support was awesome and immediately blocked the account and notified me, reseting the account after I had changed the keys."
If you're wondering how they were able to harvest GitHub commits so quickly, just read the article linked in that thread [2]. Basically you have bots drinking from GitHub's events firehose and the GHTorrent project. Every commit is monitored and harvested for passwords on the fly.
[1] https://news.ycombinator.com/item?id=8818035
[2] http://jordan-wright.com/blog/2014/12/30/why-deleting-sensit...