HealthCare.gov’s glitches prompt Obama to call in expert troubleshooters(washingtonpost.com)
washingtonpost.com
HealthCare.gov’s glitches prompt Obama to call in expert troubleshooters
http://www.washingtonpost.com/national/health-science/experts-working-to-fix-healthcaregov-insurance-marketplace-officials-say/2013/10/20/1e1a35ce-39b4-11e3-a94f-b58017bfee6c_story.html?hpid=z1
22 comments
Nobody spent $700M on a website. Over 3 years, the Centers for Medicare & Medicaid Services (CMS) spent $394 million in total to establish and operate the Federally Facilitated Exchanges. That includes building and staffing call centers, media outreach programs, consumer complaint tracking systems, tech support, employer support programs, all the integration and portals for the health insurance industry, integration with a dozen departments of the federal government, administrative staff, etc.
The "Federally-facilitated Exchanges IT" contract awarded to CGI Federal Inc, which is the one that includes building the Healthcare.gov website among other services, was $55 million. The numbers are directly from the Government Accountability Office's report:
http://www.gao.gov/assets/660/655291.pdf
That's still a big number, but it's over an order of magnitude less than yours. It should also be noted that not all of this money is coming from taxpayers. The GAO estimates that in fiscal year 2014, $450 million in fees will be collected from the insurance companies participating in the exchanges, and this money is directly credited to the account used to manage the FFE program.
The "Federally-facilitated Exchanges IT" contract awarded to CGI Federal Inc, which is the one that includes building the Healthcare.gov website among other services, was $55 million. The numbers are directly from the Government Accountability Office's report:
http://www.gao.gov/assets/660/655291.pdf
That's still a big number, but it's over an order of magnitude less than yours. It should also be noted that not all of this money is coming from taxpayers. The GAO estimates that in fiscal year 2014, $450 million in fees will be collected from the insurance companies participating in the exchanges, and this money is directly credited to the account used to manage the FFE program.
Nope, both sides are spinning like crazy here. 55 Million is a very old number.
NBC News has released a report on what the actual numbers are. It's $196 Million as of 5/15/2013, and the price appears to be well on its way to $292 Million: http://www.nbcnews.com/video/nightly-news/53309356/
That was five months before the failed launch and the emergency overhaul that's now underway, so it looks like the final number could be even higher.
NBC News has released a report on what the actual numbers are. It's $196 Million as of 5/15/2013, and the price appears to be well on its way to $292 Million: http://www.nbcnews.com/video/nightly-news/53309356/
That was five months before the failed launch and the emergency overhaul that's now underway, so it looks like the final number could be even higher.
> It should also be noted that not all of this money is coming from taxpayers [...] $450 million in fees will be collected from the insurance companies participating in the exchanges
The ACA was upheld in the Supreme Court on the basis that it was a tax. Unlike other taxes, which are paid to the government, you will be paying the tax to an insurance provider. They, in turn, will be using the money to partially fund the exchanges. So technically you're correct, but it's just a redirection of tax dollars.
The ACA was upheld in the Supreme Court on the basis that it was a tax. Unlike other taxes, which are paid to the government, you will be paying the tax to an insurance provider. They, in turn, will be using the money to partially fund the exchanges. So technically you're correct, but it's just a redirection of tax dollars.
Insurance companies are not required to participate in the exchanges (and many don't). And while people are required to buy insurance, they aren't required to buy it from the exchanges (many will still get insurance via their employers, for example).
Both sides are choosing to use the exchanges rather than one of their alternatives, so the fees paid are user fees, not disguised taxes (unless you consider all user fees disguised taxes, which is a separate philosophical discussion).
Both sides are choosing to use the exchanges rather than one of their alternatives, so the fees paid are user fees, not disguised taxes (unless you consider all user fees disguised taxes, which is a separate philosophical discussion).
I'm pretty sure the tax part upheld by the Supreme Court had to do with the individual mandate. They weren't saying that the insurance companies collect a tax, but rather that if you don't have insurance, the government can collect a tax.
Thanks for the clarification ... I've seen lots of numbers but you're right that it's not really fair to include some of the other services. On the other hand, is the failure of the website making the call centers irrelevant (or otherwise increasing the effective waste)?
I shouldn't have focused on any specific number as I'm really more interested in the huge failure in both the PM and execution of the software project. And I'd like to see contractors that fail miserably on one project removed from the list of companies that the government can use.
EDIT: I've seen a lot of companies create more complex software at a small fraction of that price ... and many of those companies exited well below $55M with smiles on their faces.
I shouldn't have focused on any specific number as I'm really more interested in the huge failure in both the PM and execution of the software project. And I'd like to see contractors that fail miserably on one project removed from the list of companies that the government can use.
EDIT: I've seen a lot of companies create more complex software at a small fraction of that price ... and many of those companies exited well below $55M with smiles on their faces.
I'm pretty sure I could get a team of about 15-20 together for half that price to build it better over the course of a year.
[deleted]
Wait... what company originally wrote the code? I'd like to know.
And I'm assuming said company billed by the hour? And of course, there's no way for the government to claw back the money?
And I'm assuming said company billed by the hour? And of course, there's no way for the government to claw back the money?
There is a lack of information here to correctly apportion blame between the government and the contractor here. While the general goals are certainly technically achievable, there are 44,000 pages of regulations generated from the affordable care act, in addition to the over 1,000 pages in the law itself. Just to dig through that and pull out everything related to insurance policy offering is going to take some serious time. In addition, each state has unique health care laws and regulations regarding the offer of insurance in that state.
However, the biggest problem I have heard of is simply stupid requirements from the government. Rather than have an open market where an individual insurance customer or any curious person could look at all of the products and prices, they make you verify your identity first. Why would they do this? They were afraid that people would get sticker shock from looking at insurance prices that didn't include the money they would be getting from taxpayers to defray the cost. This lack of trust in us has caused a lack of trust in their system.
It would seem that one thing that independent developers that want to help out could do would be to extract all of the useful data from the site. Gather all of the data about the plans and the pricing, and then make that available in a static searchable, filterable, browsable fashion without having to log in. Then, once people were able to do that, they could then go through the arduous identity verification process to qualify for a subsidy and get one of the plans. If it takes to long to crawl it out of the system, maybe you should file a FOIA request?
Once you get a basic information system up and running, and ad funded, you could then hook it up to connect you to the live site.
However, the biggest problem I have heard of is simply stupid requirements from the government. Rather than have an open market where an individual insurance customer or any curious person could look at all of the products and prices, they make you verify your identity first. Why would they do this? They were afraid that people would get sticker shock from looking at insurance prices that didn't include the money they would be getting from taxpayers to defray the cost. This lack of trust in us has caused a lack of trust in their system.
It would seem that one thing that independent developers that want to help out could do would be to extract all of the useful data from the site. Gather all of the data about the plans and the pricing, and then make that available in a static searchable, filterable, browsable fashion without having to log in. Then, once people were able to do that, they could then go through the arduous identity verification process to qualify for a subsidy and get one of the plans. If it takes to long to crawl it out of the system, maybe you should file a FOIA request?
Once you get a basic information system up and running, and ad funded, you could then hook it up to connect you to the live site.
> they make you verify your identity first. Why would they do this? They were afraid that people would get sticker shock from looking at insurance prices that didn't include the money they would be getting from taxpayers to defray the cost.
There are subsidy calculators out there that ask a few basic questions about income levels and you can get this information without requiring somebody's exact identity.
The cynic in me suggests that this system was specifically designed to get somebody's identity because at some point this information is going to get funneled directly to the IRS with the goal of going after and hassling people who still can't afford healthcare.
There are subsidy calculators out there that ask a few basic questions about income levels and you can get this information without requiring somebody's exact identity.
The cynic in me suggests that this system was specifically designed to get somebody's identity because at some point this information is going to get funneled directly to the IRS with the goal of going after and hassling people who still can't afford healthcare.
"While the general goals are certainly technically achievable, there are 44,000 pages of regulations generated from the affordable care act, in addition to the over 1,000 pages in the law itself. Just to dig through that and pull out everything related to insurance policy offering is going to take some serious time. In addition, each state has unique health care laws and regulations regarding the offer of insurance in that state."
And don't forget that the "law" has changed literally every single week via policy changes from HHS.
I was a government contractor for years and found it miserable enough.. but our milestones and requirements weren't constantly changing. I feel sorry for some of these guys.
And don't forget that the "law" has changed literally every single week via policy changes from HHS.
I was a government contractor for years and found it miserable enough.. but our milestones and requirements weren't constantly changing. I feel sorry for some of these guys.
Whenever it comes to anything to do with the government, I hear the "44,000 pages of regulations" excuse all the time.
But even if that was the case, there is no excuse for not having an adequate architecture to at least let people register for an account.
It just seems like common sense to me that if someone gave me a 44K page rule book on how to build a site that serves millions of people, the first thing I would worry about is how to, at the bare minimum, get people through the door.
We can't keep letting them get away with the 44K pages of regulation bullshit excuse.
But even if that was the case, there is no excuse for not having an adequate architecture to at least let people register for an account.
It just seems like common sense to me that if someone gave me a 44K page rule book on how to build a site that serves millions of people, the first thing I would worry about is how to, at the bare minimum, get people through the door.
We can't keep letting them get away with the 44K pages of regulation bullshit excuse.
They can't afford you for free. It would cost them so much more to vet you than using one of their usual "pre-approved" government contractors, it wouldn't even be worth their considering. Even if that could be done, no one would be willing to take responsibility for giving you access. The system is designed (or has evolved) specifically to diffuse this sort of responsibility. It would be impossible to focus it sharply enough to allow you to do this.
The code is broken, but not the broken part of this system.
The code is broken, but not the broken part of this system.
I made the offer with the assumption that there were three or four reasons it wouldn't be taken ... you've described one of them. The others are:
b) The selected contractor won the work as payback for some political benefit, and the person who made sure they won can't allow them to be perceived as incompetent.
c) They don't believe it's really broken so much as it just needs to be "tuned" (almost "the myth of sunk costs").
d) No one in the government cares ... so they're not reading our thoughts and comments on the system.
b) The selected contractor won the work as payback for some political benefit, and the person who made sure they won can't allow them to be perceived as incompetent.
c) They don't believe it's really broken so much as it just needs to be "tuned" (almost "the myth of sunk costs").
d) No one in the government cares ... so they're not reading our thoughts and comments on the system.
Reading your comments, I can really sense your frustration because I feel the same way. I live 15 minutes from California's state capital and this whole fiasco makes me want to just drive up there and do something.
They can't afford you for free. It would cost them so much more to vet you than using one of their usual "pre-approved" government contractors
Why don't they vet the results instead of vetting the people? Isn't that what matters? Not even the Soviets believed that the grain imports from the US were poisoned in order to kill them.
Why don't they vet the results instead of vetting the people? Isn't that what matters? Not even the Soviets believed that the grain imports from the US were poisoned in order to kill them.
If you want to know why no one in the federal government will take you up on your offer you should read about the Anti-Deficiency Act:
http://en.wikipedia.org/wiki/Antideficiency_Act http://www.gao.gov/legal/lawresources/antideficiencybackgrou...
It was created because of tricks and workarounds past administrations have used to get around the legislative appropriations process. One of its provisions prohibits federal employees from accepting voluntary services for the government. From what I understand, the idea was to prevent people from volunteering today with the expectation of pressuring Congress to pay them tomorrow.
And, to be clear, if it were legal I'd already be trying to find ways I could help.
http://en.wikipedia.org/wiki/Antideficiency_Act http://www.gao.gov/legal/lawresources/antideficiencybackgrou...
It was created because of tricks and workarounds past administrations have used to get around the legislative appropriations process. One of its provisions prohibits federal employees from accepting voluntary services for the government. From what I understand, the idea was to prevent people from volunteering today with the expectation of pressuring Congress to pay them tomorrow.
And, to be clear, if it were legal I'd already be trying to find ways I could help.
It makes a lot of sense, there are lots of things that you could do as a friendly helper that would steer the long term direction of an agency or program towards a particular vendor.
If you were a Linux specialist, for instance and setup some system that ran solely on Linux, you would in effect be picking a platform and preventing vendors like Microsoft and IBM from competing.
Obviously, paying some government contractor bazillions of dollars doesn't necessarily result in a great outcome, but these rules are there to stop the casual corruption that impacts the faith of the public in government.
If you were a Linux specialist, for instance and setup some system that ran solely on Linux, you would in effect be picking a platform and preventing vendors like Microsoft and IBM from competing.
Obviously, paying some government contractor bazillions of dollars doesn't necessarily result in a great outcome, but these rules are there to stop the casual corruption that impacts the faith of the public in government.
these rules are there to stop the casual corruption that impacts the faith of the public in government
Because formal, institutionalized corruption doesn't breed lack of faith, I'm sure.
Because formal, institutionalized corruption doesn't breed lack of faith, I'm sure.
Unfortunately, the government won't accept volunteers. It's really too bad, but everything has to be set in stone in a contract, and that means that the people who are already "in" are usually the only ones who can navigate the byzantine contract insanity.
Is there an executive summary of the architecture and technologies?
Something like this doesn't need more grandstanding, it's either good architecture with some bugs and scaling issues which are probably solvable or it's crap. Likewise is there some sort of jacked up insurance industry rules engine?
Something like this doesn't need more grandstanding, it's either good architecture with some bugs and scaling issues which are probably solvable or it's crap. Likewise is there some sort of jacked up insurance industry rules engine?
I wanted to send you a quick note that I'd be willing to help, on the off chance that you did manage to get that open source project going with government cooperation.
Unfortunately, you do not have an email in your profile. And your website is throwing a 503 error.
Unfortunately, you do not have an email in your profile. And your website is throwing a 503 error.
Fixed and fixed!
And thanks for pointing out the issue with my server (I'm in the process of replacing it!).
And thanks for pointing out the issue with my server (I'm in the process of replacing it!).
While the server-side components may never become open-source or auditable (significant portions are based on existing, closed systems), the client-side code is easily auditable. If you have a chance, I would love to see what you would find by analyzing https://github.com/STRML/Healthcare.gov-Marketplace. If you get a chance to look at it, please submit a PR.
The website apparently has over 500 million lines of Java code though. Nobody can cover all of that by themselves in 2 days. Undoing bad design decisions at this scale takes a LOT of work, a lot more than you are estimating. Even if there were 100 others like you, I doubt you guys could even put a dent in the problem.
Where do you get this number from?
LOL, I'm sure 2 days from the right guy are all that's required here.
Politics were allowed to trump project management.
“One highly unusual decision, reached early in the project, proved critical: the Medicare and Medicaid agency assumed the role of project quarterback, responsible for making sure each separately designed database and piece of software worked with the others, instead of assigning that task to a lead contractor.
Some people intimately involved in the project seriously doubted that the agency had the in-house capability to handle such a mammoth technical task of software engineering while simultaneously supervising 55 contractors. An internal government progress report in September 2011 identified a lack of employees ‘to manage the multiple activities and contractors happening concurrently’ as a ‘major risk’ to the whole project” [1].
It is speculated that "facing intense opposition from congressional Republicans, the administration was in a bunker mentality as it built the enrollment system, one former administration official said. Officials feared that if they called on outsiders to help with the technical details of how to run a commerce website, those companies could be subpoenaed by Hill Republicans, the former aide said” [2].
[1] http://www.nytimes.com/2013/10/13/us/politics/from-the-start...
[2] http://www.politico.com/story/2013/10/obamacare-aca-website-...
“One highly unusual decision, reached early in the project, proved critical: the Medicare and Medicaid agency assumed the role of project quarterback, responsible for making sure each separately designed database and piece of software worked with the others, instead of assigning that task to a lead contractor.
Some people intimately involved in the project seriously doubted that the agency had the in-house capability to handle such a mammoth technical task of software engineering while simultaneously supervising 55 contractors. An internal government progress report in September 2011 identified a lack of employees ‘to manage the multiple activities and contractors happening concurrently’ as a ‘major risk’ to the whole project” [1].
It is speculated that "facing intense opposition from congressional Republicans, the administration was in a bunker mentality as it built the enrollment system, one former administration official said. Officials feared that if they called on outsiders to help with the technical details of how to run a commerce website, those companies could be subpoenaed by Hill Republicans, the former aide said” [2].
[1] http://www.nytimes.com/2013/10/13/us/politics/from-the-start...
[2] http://www.politico.com/story/2013/10/obamacare-aca-website-...
[2] doesn't make sense to me.
> "Officials feared that if they called on outsiders to help with the technical details"
Aren't the 55 contractors outsiders? This sounds like typical DC finger pointing, blame game, and the-buck-doesn't-stop-here tactics.
> "Officials feared that if they called on outsiders to help with the technical details"
Aren't the 55 contractors outsiders? This sounds like typical DC finger pointing, blame game, and the-buck-doesn't-stop-here tactics.
It's washington, politics are part of project management. A 'bunker mentality' WRT subpoenas is totally justified on this project, given how Republicans have acted. You want to pay people by the hour to give testimony while their job goes undone?
As for having medicare/medicaid quarterback it.. honestly, that makes sense to me too. It's healthcare in the government, it involves deep integration with medicare/medicaid, if they're not capable of managing those contractors then they need to learn how.
I'm not defending the execution in general, but those 2 specific decisions seem ok to me. There weren't a lot of political problems building the site, so those decisions paid off -- they just needed to execute better on the technical stuff.
As for having medicare/medicaid quarterback it.. honestly, that makes sense to me too. It's healthcare in the government, it involves deep integration with medicare/medicaid, if they're not capable of managing those contractors then they need to learn how.
I'm not defending the execution in general, but those 2 specific decisions seem ok to me. There weren't a lot of political problems building the site, so those decisions paid off -- they just needed to execute better on the technical stuff.
Sounds like a standard run-of-the-mill Government contract job to me. It's a known fact getting a Government contract is like winning the lottery for a lot of businesses. Ramp up your usual price by 1000%, take twice as long as you should and when SHTH proclaim the problem is a lot more complex and bigger than you thought and all is forgiven. Having said that, not all contractors deliberately waste money and drag out projects, but it is a known aspect of contract work. Whether or not that is the reason in this instance has not been proven and probably never will be.
The Obama Care fiasco continues, almost like it was deliberately on purpose in a way.
The Obama Care fiasco continues, almost like it was deliberately on purpose in a way.
So, the fiasco continues.
A cynical person might think first contractor failed on purpose, so that the second bunch of "experts" could be brought in to fix it. One system for the price of two (or more, "expert troublshooters" don't come cheap). I'd really like to see a transparent accounting of who is getting paid, and how much, to implement this so-called website.
A cynical person might think first contractor failed on purpose, so that the second bunch of "experts" could be brought in to fix it. One system for the price of two (or more, "expert troublshooters" don't come cheap). I'd really like to see a transparent accounting of who is getting paid, and how much, to implement this so-called website.
Having worked on big government projects with big consulting companies, that's pretty unlikely. Government projects are usually undersold in order to win the tender and are subsequently delivered late and to a low standard, the difference is that they're rarely high-profile enough to get noticed by anyone who makes procurement decisions, and so the consulting companies' salespeople get to bid for the next one.
I've seen program managers getting pretty nervous when senior bureaucrats start breathing down their neck - I'd have to admire the balls on one who intentionally fucks up a project while being personally scrutinised by the President of the United States.
I've seen program managers getting pretty nervous when senior bureaucrats start breathing down their neck - I'd have to admire the balls on one who intentionally fucks up a project while being personally scrutinised by the President of the United States.
I'm perpetually puzzled that people in tech keep seeing nefarious purposes behind this failure when it sounds like every failed project we've heard of. Moving targets (how many states, how many users), feature creep, managers with no understanding of the undertaking, too many managers, and no time for testing and debugging.
It's not evil, it's just incompetent and embarrassing.
It's not evil, it's just incompetent and embarrassing.
> It's not evil, it's just incompetent and embarrassing.
And anyone who has worked in the industry over a long period of time will know that for big projects this closer to the norm than the exception.
I've personally worked on one massive banking project in the 90s that turned into a billion (in today’s terms) dollar right-off and in that time I read of many other such disasters.
And anyone who has worked in the industry over a long period of time will know that for big projects this closer to the norm than the exception.
I've personally worked on one massive banking project in the 90s that turned into a billion (in today’s terms) dollar right-off and in that time I read of many other such disasters.
Agreed. While it sounds cool to bring in a "dream team" of technical experts to save the day, it seems likely that additional non-technical issues are at play besides (or in addition to) the technical incompetence of the contractor.
Here's an alternate scenario:
1. The government agency that hired the contractors does not completely understand the business requirements in a way that allows them to effectively communicate them to the contractor. This could be due to numerous factors outside their control such as changing legislation or perhaps because the ACA is so large (900+ pages) and relatively new that there are few experts who understand the minute details sufficiently to specify how the exchange should work.
2. New requirements emerge at the last minute, requiring the contractor to make significant changes. While it is recognized that additional schedule time would be needed to sufficiently implement and test the required features, there is enormous pressure (political and otherwise) to adhere to the October 1 release date, so the implementation of the features is rushed and QA is skipped.
3. There are dependencies on legacy government and commercial systems that are outside the control of the contracting agency and the contractor. The functionality of these systems impact reliability and performance, introduce strange bugs, etc.
5. The software is delivered on time but is woefully inadequate.
While I wouldn't rule out the incompetence of the technical staff, these types of issues can often be traceable to problems that are more in the realm of management.
Here's an alternate scenario:
1. The government agency that hired the contractors does not completely understand the business requirements in a way that allows them to effectively communicate them to the contractor. This could be due to numerous factors outside their control such as changing legislation or perhaps because the ACA is so large (900+ pages) and relatively new that there are few experts who understand the minute details sufficiently to specify how the exchange should work.
2. New requirements emerge at the last minute, requiring the contractor to make significant changes. While it is recognized that additional schedule time would be needed to sufficiently implement and test the required features, there is enormous pressure (political and otherwise) to adhere to the October 1 release date, so the implementation of the features is rushed and QA is skipped.
3. There are dependencies on legacy government and commercial systems that are outside the control of the contracting agency and the contractor. The functionality of these systems impact reliability and performance, introduce strange bugs, etc.
5. The software is delivered on time but is woefully inadequate.
While I wouldn't rule out the incompetence of the technical staff, these types of issues can often be traceable to problems that are more in the realm of management.
all the states. all the citizens. how is that not obvious? these are known knowns.
No it's not.
All states were supposed to have their own exchanges. 14 ended up doing it. New York after all sorts of political machinations agreed to run their own exchange at the 11th hour. (Supposedly "unprecedented" traffic also screwed up the NY exchange, which was also built by some big government contracting firm -- CSC)
All states were supposed to have their own exchanges. 14 ended up doing it. New York after all sorts of political machinations agreed to run their own exchange at the 11th hour. (Supposedly "unprecedented" traffic also screwed up the NY exchange, which was also built by some big government contracting firm -- CSC)
i thought healthcare.gov is for those states which opted out of the medicaid expansion (i'm not sure about this). it may have been the case that the number which have opted out was unexpected by an order of magnitude. i don't know if i'd buy this excuse though
I don't think the cynical view makes a lot of sense. The original contractor is likely billing for hours to fix the site now. Bringing in additional experts doesn't benefit them. A failure of this scale and visibility isn't going to help their business reputation and if they lose the maintenance contract on the site they're likely to be out for millions. I can see bidding low with the expectation of making a killing on change orders (I think the contractor succeeded at that), but outright failure to deliver doesn't make a lot of sense.
I think like most IT projects you need to look at the management team in charge and the decisions they made to understand if this failure could have been identified and avoided.
I think like most IT projects you need to look at the management team in charge and the decisions they made to understand if this failure could have been identified and avoided.
> A failure of this scale and visibility isn't going to help their business reputation
Do big failures really hurt people or companies all that much? At some point, you're dealing in a realm in which relatively few people even play (or perhaps even want to play) and you may be the only game in town anyway.
Bankers/CEOs who lost millions were still given large bonuses with the reasoning that "we wouldn't be able to find anyone else to do their job". Well... I could certainly lose millions faster than many of them, and I'd do it for half their bonus, thereby saving the companies money :)
Do big failures really hurt people or companies all that much? At some point, you're dealing in a realm in which relatively few people even play (or perhaps even want to play) and you may be the only game in town anyway.
Bankers/CEOs who lost millions were still given large bonuses with the reasoning that "we wouldn't be able to find anyone else to do their job". Well... I could certainly lose millions faster than many of them, and I'd do it for half their bonus, thereby saving the companies money :)
"Do big failures really hurt people or companies all that much?"
Some folks do get black balled[1], and companies will liquidate from time-to-time, but in the end, I really cannot think of anyone that didn't end up richer except the poor souls the companies hired. These days, most of them just switch to another company and continue the cycle.
1) if your name is know to a career, high-level administrator in a bad way, expect a lack of contracts
Some folks do get black balled[1], and companies will liquidate from time-to-time, but in the end, I really cannot think of anyone that didn't end up richer except the poor souls the companies hired. These days, most of them just switch to another company and continue the cycle.
1) if your name is know to a career, high-level administrator in a bad way, expect a lack of contracts
That's something I used to wonder too. I used to work for a company that had a number of highly-visible failures, the kind that gets reported in the news. They somehow still managed to score multi-million-dollar contracts. I suspect having a great sales team is more important that having a great engineering team.
No point really. It's taxpayer money. It's limitless.
Plus the Republican missteps on the shutdown/debt ceiling means that their best anti-Obamacare story barely got any airtime.
And since there wasn't much airtime, there won't be much pressure to get to the bottom of it.
And since there wasn't much airtime, there won't be much pressure to get to the bottom of it.
this.
But to play Government's advocate a little: That's also kind of how we got awesome things like GPS systems kinda fast.
But to play Government's advocate a little: That's also kind of how we got awesome things like GPS systems kinda fast.
You do not want to embarrass an administrator as a contractor[1]. If there is any rule of government contracting, its that one. Large military contractors can get away with it because they are few and have much lobbying money. Smaller contractors should know better. I would be curious if this company dissolves in the future. I remember a company in the 90's that ticked off someone in HHS and the company was no more in about a year (had a really generic name with a common TLA, my notes are in a storage locker).
1) which should make you read very closely some government findings when it was a pet project of some longtime admin or Senator.
1) which should make you read very closely some government findings when it was a pet project of some longtime admin or Senator.
Does anyone have more information about what exactly is going wrong with this site? Apparently Development Seed (the company behind http://mapbox.com) was (one of?) the contractors behind the project[0] and they've always seemed (at least judging from their product offerings) quite competent.
0. http://developmentseed.org/blog/2013/06/25/healthcare-launch...
0. http://developmentseed.org/blog/2013/06/25/healthcare-launch...
Dev Seed did the user-information/knowledge-base component, which has worked fine. They didn't work on the healthcare exchange, which is what has been broken.
IIRC, the site was handed out to 55 (!) contractors to create?
Maybe #56 is the charm.
I take no pleasure in watching this train wreck. It could, however, be a teaching moment for the country on what's wrong with Federal IT. Probably won't be, but it could be. So the story is worth following.
Maybe #56 is the charm.
I take no pleasure in watching this train wreck. It could, however, be a teaching moment for the country on what's wrong with Federal IT. Probably won't be, but it could be. So the story is worth following.
I think 3 really good programmers, 2 sys admins, 1 non-clueless PM, and 1 person to arrange their travel would have gotten this done faster.
No - they'd by now have read through about 20% of the regulations the system needs to fulfill.
What a pile of BS.
They're so moronic that they built the thing such that registration of the userbase would take more than a hundred years.
Stop coddling the state, they're not going to do it right, they don't welcome the debate, they don't care, they won't present a balanced budget, the debt default is constitutionally prohibited, the war on drugs is a war on people, their wars are so obviously wrong that there are more soldier suicides than deaths in combat, the military burns more oil than the entire population of india, socialism doesn't suddenly work when the current regime tries it.... Oh never mind, go ahead and keep wasting your time listening to sociopaths and participating in silly, futile politics; I'll just make a greasemonkey script to filter out all stories with "Obama" in them, and be on my merry way.
They're so moronic that they built the thing such that registration of the userbase would take more than a hundred years.
Stop coddling the state, they're not going to do it right, they don't welcome the debate, they don't care, they won't present a balanced budget, the debt default is constitutionally prohibited, the war on drugs is a war on people, their wars are so obviously wrong that there are more soldier suicides than deaths in combat, the military burns more oil than the entire population of india, socialism doesn't suddenly work when the current regime tries it.... Oh never mind, go ahead and keep wasting your time listening to sociopaths and participating in silly, futile politics; I'll just make a greasemonkey script to filter out all stories with "Obama" in them, and be on my merry way.
Why are there no consequences for this epic failure? Someone should go to jail for lying about the costs and the schedule. Many more should get fired and no longer permitted to work on government projects.
Why are there no consequences for this epic failure?
Because government. They get paid the same, by you and me, whether it works or not.
Because government. They get paid the same, by you and me, whether it works or not.
This probably doesn't really qualify as a failure yet in government contracting. People haven't been fired for much more twisted projects, like the F35.
What is the point of healthcare.gov anyways? You can go and buy an ACA compliant plan on a comparison shop website like ehealthinsurance.com that works pretty well why did the government need to create it's own?
I think you can only access the subsidies if you buy through healthcare.gov
The idea was to attack fragmentation, which is a huge cost center in the health industry. A smaller individual insurance plan
* Has to pay more for drugs (less bargaining power vs a big plan)
* Has more statistical risk (# people goes with N, stdev in # people with a given illness goes with sqrt(N), so risk/person goes with sqrt(N)/N and larger plans have less risk)
* Has larger legal/consulting overhead
* Forces doctors to incur overhead with nonstandard forms
* Is vulnerable to the bandwagon effect (if, by accident, it happens to favor X group of costly individuals, the costly individuals proceed to pile on and sink the plan)
* Has the ability to discriminate based on fine-grained geographical knowledge and other factors
* Has an under-leveraged customer base (if the plan is "secretly evil" and dumps people who need expensive treatment and 1% of people require expensive treatment, a 100 person plan will have 0 or maybe 1 person being screwed at any given time, while a 10000 person plan will have ~100 people being screwed who are much more able to band together and fight a legal/PR battle)
Previously, small plans were able to flourish because of #6 and #7, incurring #1-5 as costs of doing business. This was bad for everyone except insurance companies. The best way to turn a profit in the insurance industry is to sell insurance priced for a medium-risk person to a low-risk individual(or hi-risk price to a medium-risk person). If you look closely, this kind of "innovation" is actually just restoring the pre-insurance reality of charging sick people a lot and healthy people very little. It defeats the purpose of insurance while increasing costs (#1-5) and letting the people in charge of the purpose-defeating process pocket a hefty cut of the proceeds by fooling people into buying overpriced coverage.
Exchanges are designed to put an end to the tragedy of the commons. The savings from #1-5 can be recouped at the expense of eliminating #6 and #7, which were perverse incentives anyway.
Even if you disagree with my explanations, it is objectively true that employer plans are much more efficient than individual plans (E(payout)/E(cost) is larger). You can then see the ACA exchanges as an attempt to reform individual plans to emulate employer plans. Of course, it remains "cargo cult economics" unless you can figure out why you expect the emulation to close the gap in value, which is what I was doing with the list in the first place. In other words: I have provided 7 possible explanations that I think make sense but there are even more out there which I don't agree with and haven't listed.
* Has to pay more for drugs (less bargaining power vs a big plan)
* Has more statistical risk (# people goes with N, stdev in # people with a given illness goes with sqrt(N), so risk/person goes with sqrt(N)/N and larger plans have less risk)
* Has larger legal/consulting overhead
* Forces doctors to incur overhead with nonstandard forms
* Is vulnerable to the bandwagon effect (if, by accident, it happens to favor X group of costly individuals, the costly individuals proceed to pile on and sink the plan)
* Has the ability to discriminate based on fine-grained geographical knowledge and other factors
* Has an under-leveraged customer base (if the plan is "secretly evil" and dumps people who need expensive treatment and 1% of people require expensive treatment, a 100 person plan will have 0 or maybe 1 person being screwed at any given time, while a 10000 person plan will have ~100 people being screwed who are much more able to band together and fight a legal/PR battle)
Previously, small plans were able to flourish because of #6 and #7, incurring #1-5 as costs of doing business. This was bad for everyone except insurance companies. The best way to turn a profit in the insurance industry is to sell insurance priced for a medium-risk person to a low-risk individual(or hi-risk price to a medium-risk person). If you look closely, this kind of "innovation" is actually just restoring the pre-insurance reality of charging sick people a lot and healthy people very little. It defeats the purpose of insurance while increasing costs (#1-5) and letting the people in charge of the purpose-defeating process pocket a hefty cut of the proceeds by fooling people into buying overpriced coverage.
Exchanges are designed to put an end to the tragedy of the commons. The savings from #1-5 can be recouped at the expense of eliminating #6 and #7, which were perverse incentives anyway.
Even if you disagree with my explanations, it is objectively true that employer plans are much more efficient than individual plans (E(payout)/E(cost) is larger). You can then see the ACA exchanges as an attempt to reform individual plans to emulate employer plans. Of course, it remains "cargo cult economics" unless you can figure out why you expect the emulation to close the gap in value, which is what I was doing with the list in the first place. In other words: I have provided 7 possible explanations that I think make sense but there are even more out there which I don't agree with and haven't listed.
You forgot one of the biggest reasons for the exchanges: they're the mechanism through which the government provides premium subsidies to people who qualify.
I think there are a variety of reasons for that (e.g. keeping better tabs on the large amount of money spent on subsidies, easier administration of the subsidy tax credit), but the bottom line is the law says people only get the subsidies if they go through an exchange, so...
I think there are a variety of reasons for that (e.g. keeping better tabs on the large amount of money spent on subsidies, easier administration of the subsidy tax credit), but the bottom line is the law says people only get the subsidies if they go through an exchange, so...
The government has many mechanisms through which it distributes subsidies, and while I'm sure the centralized exchange is going to be one of the most efficient and convenient, I'm not nearly so certain that the difference will be larger than the effects it has on the market.
Let me be concrete. According to [1] (which gets its numbers from the CBO), individual plans have 29% administrative overhead as opposed to 12% for employer plans. If the exchanges successfully emulate employer plans, they save 17%*$300b = $51b per year in health care costs. Unless the exchange program is 10,000% more efficient than its alternatives (tax subsidy, payments to companies, etc) then the effects I listed dominate.
In any case, the exchanges were designed with the explicit intent of making the individual insurance market more efficient, not merely making subsidy disbursement more efficient.
http://www.americanprogressaction.org/issues/healthcare/repo...
Let me be concrete. According to [1] (which gets its numbers from the CBO), individual plans have 29% administrative overhead as opposed to 12% for employer plans. If the exchanges successfully emulate employer plans, they save 17%*$300b = $51b per year in health care costs. Unless the exchange program is 10,000% more efficient than its alternatives (tax subsidy, payments to companies, etc) then the effects I listed dominate.
In any case, the exchanges were designed with the explicit intent of making the individual insurance market more efficient, not merely making subsidy disbursement more efficient.
http://www.americanprogressaction.org/issues/healthcare/repo...
A very good point. The federal government was supposed to have states create their own exchanges, but yes, this it seems like the commercial space could handle some of this.
Many of the states refused to create their own exchanges, thus the Federal site needed to be more extensive than originally planned (not that's an excuse for the screw-ups at all). States that have their own exchanges seem to be doing better, from anecdotal evidence supplied so far…
It is an interesting question though of why we need a federal site in the first place in order to shop for a commercial product. We are required to have car insurance but we don't need the state government running a car insurance shopping site. I am not for/against the Affordable Care Act, but I think it is a legitimate question.
Well the answer is that government sites will determine subsidies. That includes verifying income and checking with numerous federal agencies. http://www.washingtonpost.com/blogs/the-switch/wp/2013/10/09...
requirements for car insurance are set by each state, IIRC. and not everyone has a car - it's not mandatory. Everyone has a ... 'health'.
What I am more worried about than any other HealthCare.Gov glitch, is if this website was so badly designed, that the data it holds, the functions it does would also be extremely badly designed and secured. Yet NO ONE is talking about this angle.
Thats what keeps me up at night.
Thats what keeps me up at night.
What did anyone really expect? Sounds like a typical government contract.
I can only imagine how these conversations go...
"We need more QA! STAT! It's going to take at least $50 million to test this system... it's awfully broken!" chuckle
face/palm
That said, I think there should be a lot more concern about the security of this site if there's been such a mishandling of the basic functions and operational flow.
Seems only reasonable to suspect this will be leading to a horrible leak of many people's private records at some near point in the future.
I can only imagine how these conversations go...
"We need more QA! STAT! It's going to take at least $50 million to test this system... it's awfully broken!" chuckle
face/palm
That said, I think there should be a lot more concern about the security of this site if there's been such a mishandling of the basic functions and operational flow.
Seems only reasonable to suspect this will be leading to a horrible leak of many people's private records at some near point in the future.
This one doesn't even seem that bad yet. I mean its not like the new bay bridge which was a decade late.
Obama didn't read The Mythical Man-Month.
(Chuckle) Well seeing as how half the tech managers I talk to at private companies haven't ever heard of it, I think he should be given a little bit of a break.
He obviously had - his campaign was praised about the high quality of IT personal and solutions that were developed.
But after all getting reelected is more important than making sure your signature piece of legislation is actually working at launch.
But after all getting reelected is more important than making sure your signature piece of legislation is actually working at launch.
The solution was built in 2/3rds by Accidenture, who got awared a $400m contract (out of the total of $600m awarded) with the bulk of the remainder going to IBM and the 50 other contractors.
If I know one thing, and I say this from firsthand experience, it's that Accidenture is absolutely the worst choice for any kind of contract.
Accidenture isn't an "outsourcing" company. They're an oversized roleplaying project, where the overlords (controllers) have the power and whim to kill projects and just pay the termination penalty, and where everybody strives to become a partner because it's up or out. Any problem inside Accidenture is seen as a nail, and every solution is a hammer, meaning a combination of ISS, MSSQL, Sharepoint, Windows Server and copious application of ridiculously overpriced "consultants".
If I know one thing, and I say this from firsthand experience, it's that Accidenture is absolutely the worst choice for any kind of contract.
Accidenture isn't an "outsourcing" company. They're an oversized roleplaying project, where the overlords (controllers) have the power and whim to kill projects and just pay the termination penalty, and where everybody strives to become a partner because it's up or out. Any problem inside Accidenture is seen as a nail, and every solution is a hammer, meaning a combination of ISS, MSSQL, Sharepoint, Windows Server and copious application of ridiculously overpriced "consultants".
The magic quote:
More broadly (and apologies for the non-HN bent to this), the project serves as an example for the utter shitshow that is the United States healthcare system. The exchange is intended to serve as an abstraction for the process of figuring out what public and private options are available to you. The motivation for building the exchange was "Hey! Figuring out what options exist for healthcare is complicated!". Of course, building such a system doesn't make the complexity go away - it just makes it someone else's job. Perhaps that this determination is so complicated that it requires a ridiculous system is an indication that we're "doing it wrong".
One major problem slowing repairs, people close
to the program say, is that the Centers for Medicare
and Medicaid Services, the federal agency in charge
of the exchange, is responsible for making sure that
the separately designed databases and pieces of
software from 55 contractors work together.
More broadly (and apologies for the non-HN bent to this), the project serves as an example for the utter shitshow that is the United States healthcare system. The exchange is intended to serve as an abstraction for the process of figuring out what public and private options are available to you. The motivation for building the exchange was "Hey! Figuring out what options exist for healthcare is complicated!". Of course, building such a system doesn't make the complexity go away - it just makes it someone else's job. Perhaps that this determination is so complicated that it requires a ridiculous system is an indication that we're "doing it wrong".
I've seen this "ancient systems" idea go by a few times. Honest question: If these platforms are so old, they likely aren't terribly complex, from the perspective of what's now computationally cheap. So, how hard would it be to drop a computer on their local network and just dumb-clone the whole thing?
They could learn a lot from how UK.gov (http://digital.cabinetoffice.gov.uk/) operates these days. It's a true agile shop, that's been ridiculously productive and forward-thinking for at least the last 5 years.
So in order to reduce health care costs the government declares itself as all knowing in the medical industry, inserts itself into the system with all the beauracracy. Sounds to me like adding more middle men just raises costs and also happens to kick open the door for typical government graft, corruption and embezzlement. Better yet the government sets itself up to decide who gets treatment, and data mining by the NSA and IRS can help identify polical opponents or non supporters to determine on the fly available coverages and rates.
Perfect for any tyranical statist type.
Does anyone really believe the whole website thingy wasn't full of graft, corruption, kickbacks and paybacks? Come on, seriously.
Perfect for any tyranical statist type.
Does anyone really believe the whole website thingy wasn't full of graft, corruption, kickbacks and paybacks? Come on, seriously.
All I know is that they need to start making decisions like minimizing their css and js scripts and fetching jquery from Google before I start taking them serious when they say the best engineers in the fed are working on it.
<script src='//assets.healthcare.gov/global/js/lib/jquery-1.8.2.js'></script>
https://www.healthcare.gov/css/all.css https://www.healthcare.gov/js/all.js
<script src='//assets.healthcare.gov/global/js/lib/jquery-1.8.2.js'></script>
https://www.healthcare.gov/css/all.css https://www.healthcare.gov/js/all.js
This type of integration project is always challenging regardless whether it is government/private sector. Then add tons of required-by-law requirements and you are talking about massive tasks to get things right.
Their main problem is the hard deadline for October 1st launch. Things should have been rolled out state by state - but I am not sure whether this is in the law or just a political decision.
Their main problem is the hard deadline for October 1st launch. Things should have been rolled out state by state - but I am not sure whether this is in the law or just a political decision.
Shouldn't they have been called in a year ago? Six months ago? A month ago?
$95 Million apparently buys everything but new management and proper coders.
$95 Million apparently buys everything but new management and proper coders.
Now what? Another $600 million down the drain to fix the problems of the first $600 million?
What kind of computer programmers did the government hire to oversee the progress? I am going to question this person's ability. While I am just a senior in college, I probably can hammer a lot more practical issues before its launch. I am sure half of the HNers can do that too.
sigh
sigh
Thanks for downvoting me, government employees.
If you're lucky, you'll learn this lesson early in your career: when you think "oh, that's easy, those people must be idiots" it's a sign that you don't understand the problem.
People are not idiots. Large problems are never easy. Politics, management, and salesmanship are genuine parts of the problem, not useless distractions from technical stuff.
I'm not saying that improvement is impossible, I'm saying that improvement is hard. The healthcare.gov system could have gone live, and been functional, on October 1 - if you could hire better technical people. Can you hire a top-notch team of techies in this environment? Ever tried it? It could be working if the political appointees had nailed down the requirements earlier. Have you ever tried getting hundreds of legally-mandated decisions pushed through the federal bureaucracy? Many of those decisions were pushed back for political reasons, which sucks - but could you made the unpopular decisions early, then won enough elections to keep Republicans from dismantling the entire thing? The system could be working if the law made more sense in a dozen different ways, but all of those compromises were necessary to get the law passed in the first place. Have you ever tried to write legislation, and get it through congress?
Don't get me wrong - the healthcare.gov system is a clusterfuck, and there are easily identifiable reasons for that - in retrospect. And hopefully we can learn from the failures and move forward. But laying it all at the feet of some nebulous government employee is completely unhelpful.
People are not idiots. Large problems are never easy. Politics, management, and salesmanship are genuine parts of the problem, not useless distractions from technical stuff.
I'm not saying that improvement is impossible, I'm saying that improvement is hard. The healthcare.gov system could have gone live, and been functional, on October 1 - if you could hire better technical people. Can you hire a top-notch team of techies in this environment? Ever tried it? It could be working if the political appointees had nailed down the requirements earlier. Have you ever tried getting hundreds of legally-mandated decisions pushed through the federal bureaucracy? Many of those decisions were pushed back for political reasons, which sucks - but could you made the unpopular decisions early, then won enough elections to keep Republicans from dismantling the entire thing? The system could be working if the law made more sense in a dozen different ways, but all of those compromises were necessary to get the law passed in the first place. Have you ever tried to write legislation, and get it through congress?
Don't get me wrong - the healthcare.gov system is a clusterfuck, and there are easily identifiable reasons for that - in retrospect. And hopefully we can learn from the failures and move forward. But laying it all at the feet of some nebulous government employee is completely unhelpful.
I will, at my own expense drive to D.C. and spend two days of my vacation evaluating the architecture of the failing system, and spot check the source code quality. Additionally, I'll add the project to my Sonar source code quality analyzer so that there's a focus on what code isn't up to industry standards and which sections should be addressed first.
It's still astounding to me that we spent $700M on such a pathetic LITTLE system ... there simply isn't that much code to write. If there are others who'd like to show the government how it should be done, I'd be happy to start an open-source project to rewrite the whole project, as well as a CI/CT/CD/CM system to make sure that the code is maintainable, that changes are delivered in a reasonable amount of time and that servers are constructed and clustered in a consistent way.
This really isn't rocket science (and it's a good thing ... there would have been deaths).
EDIT: If someone in the government a) reads my offer and b) is willing to let "the industry" voluntarily help, you can contact me via the e-mail address on my profile page.