I am senior sysadmin/infrastructure guy with >15 years experience. I am currently in between side projects and I am looking for extra work.
I have designed and built the infrastructure for sites and applications that serve millions of users a day. In the past I have specialized in migrating stacks off creaking infrastructure to scalable and redundant new homes. Experienced in very high load environments.
If you are not at the stage where you need full time ops people but want to start off with a good foundation or are just seeking professional validation of your existing setup let me know.
This is a mostly solved problem though. Or at least there are people and products out there that do this specifically for you, not with a focus on bots alone as that is a bit of a edge case but for mobile devices.
DeviceAtlas and WURFL are both products that I have used that attempt to have complete coverage of all UA strings in use in the wild even, the ones that attempt to spoof legitimate UA strings.
While their focus is on device detection and the properties of these devices for mobile content adaptation, knowing which connections are from bots is just as important if you are going to alter what content you serve up based on UA.
These product actively seek out new UAs and add them to their rule sets in a way that I can only assume will be more accurate over time than a home grown solution.
FWIW I know that FB use at least one of these products for their device detection, so if you want to match what FB thinks (knows!?) this is where you would start.
The process of front running or domain tasting 'should' be a thing of the past. Last year or perhaps 2009 ICANN imposed strict fees on any registrar that pulled more than 10% of their monthly registrations in the first 5 days.
Obviously this only counts for ICANN TLDs and not ccTLDs.
So... When you register a domain the registrar has to pay the registry com/net/org etc immediately. The registry will return the registration fees for any domains that are canceled. However if a registrar has >10% or their total monthly registrations cancelled within the 5 day grace, they do not get that money back.
I agree. This is not the first time that Amazon have targeted the companies that can/could be used to add value to their offerings.
There was a company (cannot remember the same sorry) that provided MySQL in amazons cloud before they launched their RDS service. And an much better service it was too. The startup did not ask you to give them a 4 hour window where they can bring down your DB. But still it was an Amazon service so the brand awareness was there.
This seems like a direct attack on the likes of cloudkick. I do not understand the motivation to be honest. They were making some money from their own monitoring and not they are making none, but they are pushing companies like cloudkick away from survival and innovating more.
While the free micro instances offer was obviously a push to put the boot into Rackspace and attempt to take the remaining % of the IaaS market this I understand less.
It seems Amazon have ignored the old rules where in a new market there is less direct competition as there is plenty to go around and have gone straight for total market domination.
Disclosure: My project has(had) as a highlight feature IaaS monitoring, metrics and alerting. So if the above sounds a little unreasoned please forgive me while I contemplate a pivot. :-(
Email: Google Apps works fine for everything we need.
Code Management : Local Redmine install. Mutli repo and multi project is just what we need for a bunch or side projects or prototypes. Github for personal stuff. The issue tracking is not where it needs to be yet for an production project yet but getting there.
Deployments: Fabric and Chef-Solo. Internally a lot of virtualbox vm's.
EC2, Rackspace and hosteurope.de with : Haproxy lb's, powerDNS homegrown GeoDNS, AWS Cloudfront CDN.
I am not sure I understand the issue here. Sure, the AWS ELS offering is lacking and the issues re DNS are known. But just like any of their other offerings why try and shoe horn the product into your requirements. ?
Our service required a load balanced service and just like you we identified the issues and decided we could not live with the limitations.
So we brought up a small instance and run HAProxy on it to do all our load balancing. We assign an Elastic IP and we get to retain control, avoid the DNS issues, do https etc.
Essentially avoiding all the limitations of the Amazon offering.
AWS themselves use HaProxy for Elastic Load balancing and its solid.
I am senior sysadmin/infrastructure guy with >15 years experience. I am currently in between side projects and I am looking for extra work.
I have designed and built the infrastructure for sites and applications that serve millions of users a day. In the past I have specialized in migrating stacks off creaking infrastructure to scalable and redundant new homes. Experienced in very high load environments.
If you are not at the stage where you need full time ops people but want to start off with a good foundation or are just seeking professional validation of your existing setup let me know.
Contact details in my profile.