Ask HN: I am in a David vs. Goliath fight and I need some advice
6 comments
Get a lawyer is the best advice you'll get -- just like every other time someone Asks HN what to do with (potential) legal matters. Apparently, most lawyers will do some sort of initial consult for free. Also, might be a good idea to delete this post ASAP, if you're seriously considering legal action.
I have not consulted a lawyer yet, the reason being the competition rules state the following:
"By participating in the Competition, each Entrant agrees to release, indemnify and hold harmless GE, Kaggle Inc., and their respective affiliates, subsidiaries, advertising and promotions agencies, as applicable, and each of their respective agents, representatives, officers, directors, shareholders, and employees from and against any injuries, losses, damages, claims, actions and any liability of any kind resulting from or arising out of your participation in or association with the Competition. GE is not responsible for any miscommunications such as technical failures related to computer, telephone, cable, and unavailable network or server connections, related technical failures, or other failures related to hardware, software or virus, or incomplete, late or misdirected Entries. GE reserves the right to cancel, modify or suspend the Competition should any computer virus, bug or other technical difficulty or other causes beyond the control of GE corrupt the administration, security or proper play of the Competition, and to determine winners from among Entries not affected by the corruption, if any, in its sole discretion."
I interpret that as saying: GE can pick the winners using its sole discretion. So I don't believe this is a legal issue I can win. I have a lawyer in the family, I will run it by her and see what her opinion is.
"By participating in the Competition, each Entrant agrees to release, indemnify and hold harmless GE, Kaggle Inc., and their respective affiliates, subsidiaries, advertising and promotions agencies, as applicable, and each of their respective agents, representatives, officers, directors, shareholders, and employees from and against any injuries, losses, damages, claims, actions and any liability of any kind resulting from or arising out of your participation in or association with the Competition. GE is not responsible for any miscommunications such as technical failures related to computer, telephone, cable, and unavailable network or server connections, related technical failures, or other failures related to hardware, software or virus, or incomplete, late or misdirected Entries. GE reserves the right to cancel, modify or suspend the Competition should any computer virus, bug or other technical difficulty or other causes beyond the control of GE corrupt the administration, security or proper play of the Competition, and to determine winners from among Entries not affected by the corruption, if any, in its sole discretion."
I interpret that as saying: GE can pick the winners using its sole discretion. So I don't believe this is a legal issue I can win. I have a lawyer in the family, I will run it by her and see what her opinion is.
Please, get a lawyer outside the family with experience of competition law - maybe your sister/aunt can tell you the area of law that might apply and point to some local firms
You need objective advice, not coloured by "what if my advi e leads to not speaking to half the family" - don't put that on her.
Secondly, consult a lawyer because they have - if according to the common-sense rules you should get 30k (and I don't at all understand the situation) then no amount of disclaimers or indemnification deprives you - statute on competitions run for marketing purposes are widespread.
Maybe you have a case, maybe not, but without a lawyer you won't know and without a lawyer they (GE?Kaggle) won't take you seriously - which kind of screws the only other reason you seem to have for not getting a lawyer - the idea that maybe you will pick up some work later on. If they see you as a push over on this, why will they respect you next time?
This does not mean drop a writ for 10 million in the morning - get advice, if there is a case, ask if the lawyer can directly contact the organisers lawyers for clarification or discovery. If you have a case they will have to do something - push now, so that their decision has not been set in concrete and them moved on.
You need objective advice, not coloured by "what if my advi e leads to not speaking to half the family" - don't put that on her.
Secondly, consult a lawyer because they have - if according to the common-sense rules you should get 30k (and I don't at all understand the situation) then no amount of disclaimers or indemnification deprives you - statute on competitions run for marketing purposes are widespread.
Maybe you have a case, maybe not, but without a lawyer you won't know and without a lawyer they (GE?Kaggle) won't take you seriously - which kind of screws the only other reason you seem to have for not getting a lawyer - the idea that maybe you will pick up some work later on. If they see you as a push over on this, why will they respect you next time?
This does not mean drop a writ for 10 million in the morning - get advice, if there is a case, ask if the lawyer can directly contact the organisers lawyers for clarification or discovery. If you have a case they will have to do something - push now, so that their decision has not been set in concrete and them moved on.
Also, does anyone else have a problem with Kaggle offering money and points for resolving the matter unofficially?
When the offer was made, I was really surprised because it seemed unethical and unfair - both the the other competitors as well as the sponsor.
Just curious what other, non-Kaggle competitor opinions are on this topic.
When the offer was made, I was really surprised because it seemed unethical and unfair - both the the other competitors as well as the sponsor.
Just curious what other, non-Kaggle competitor opinions are on this topic.
You have a dispute with them. They're running a business and want the dispute (and you) to go away. Eventually, you will go away, either because you gave up or lost in court; or because they paid you to do so. If you're ethically entitled to compensation, this is a perfectly decent thing for them to do.
tl;dr: contestant submitted 23 entries to a contest, rules state you have to select one for official calculation of your score, contestant changed his mind a few times and ultimately selected an entry that did not have enough points to win despite having another entry that did have enough points.
--
In reading Anthony response, this seems less about a "bug" but more about you not being happy with the results and blaming the complexity of the submission and selection process.
This part of his response seems fairly clear:
"The submission page stated: “Note: You can select up to 1 submission that will be used to calculate your final leaderboard score. If you do not select them, up to 1 entry will be chosen for you based on your most recent submissions. Your final score will not be based on the same exact subset data as the public leaderboard, but rather a different private data subset of your full submission. Your public score is only a rough indication of what your final score might be. You should choose an entry that will most likely be best overall, and not necessarily just on the public subset.”
During the course of the competition, you made 23 submissions of which 3 were selected for leaderboard scoring. First you selected submission 242748 on January 7, 2013 (all times UTC). You unselected that submission on January 24, 2013 and selected submission 250852. You unselected that submission on January 26, 2013 and selected submission 252143. You did not revise that selection at the time of your final submission. It seems logical to conclude that you understood the process of selecting submissions. Three of the other winners selected their final submission for scoring. The two that did not select the final submission had not previously made a manual selection or had unchecked prior submissions and the software worked as designed and selected the final submission. One participant (but not a winner) made 3 final submissions and selected one for scoring - which was the case the software for selecting a response was intended to handle. There was no bug in the software. "
While unfortunate for you, it is worth noting that all of the competitors had to follow the same set of submission guidelines to be considered. Your response becomes a debate on how you “feel” the submission process should work, as opposed to indicating a “bug” in the software.
“What on earth is a checkbox doing on the submission screen of a two phase competition? There is no ambiguity - there is only a single file type that is valid for the final leaderboard file. Why on earth is your software not smart enough to figure that out?
Also - why didn't the default selection logic kick in? Because only final leaderboard files are valid for the final submission, why didn't your software pick the only valid option?”
--
In reading Anthony response, this seems less about a "bug" but more about you not being happy with the results and blaming the complexity of the submission and selection process.
This part of his response seems fairly clear:
"The submission page stated: “Note: You can select up to 1 submission that will be used to calculate your final leaderboard score. If you do not select them, up to 1 entry will be chosen for you based on your most recent submissions. Your final score will not be based on the same exact subset data as the public leaderboard, but rather a different private data subset of your full submission. Your public score is only a rough indication of what your final score might be. You should choose an entry that will most likely be best overall, and not necessarily just on the public subset.”
During the course of the competition, you made 23 submissions of which 3 were selected for leaderboard scoring. First you selected submission 242748 on January 7, 2013 (all times UTC). You unselected that submission on January 24, 2013 and selected submission 250852. You unselected that submission on January 26, 2013 and selected submission 252143. You did not revise that selection at the time of your final submission. It seems logical to conclude that you understood the process of selecting submissions. Three of the other winners selected their final submission for scoring. The two that did not select the final submission had not previously made a manual selection or had unchecked prior submissions and the software worked as designed and selected the final submission. One participant (but not a winner) made 3 final submissions and selected one for scoring - which was the case the software for selecting a response was intended to handle. There was no bug in the software. "
While unfortunate for you, it is worth noting that all of the competitors had to follow the same set of submission guidelines to be considered. Your response becomes a debate on how you “feel” the submission process should work, as opposed to indicating a “bug” in the software.
“What on earth is a checkbox doing on the submission screen of a two phase competition? There is no ambiguity - there is only a single file type that is valid for the final leaderboard file. Why on earth is your software not smart enough to figure that out?
Also - why didn't the default selection logic kick in? Because only final leaderboard files are valid for the final submission, why didn't your software pick the only valid option?”
That is a good summary, but it is a little bit more complex than what you have stated. But your comment is helpful because this is exactly the understanding that Goldbloom has and it is incorrect in an important way.
The 23 entries that were submitted are for the public leaderboard set. The purpose of this set is so over the duration of the competition, competitors can see how they are performing relative to each other. These 23 submissions have no impact on the final prize winners.
A different data set, called the final data set is what actually determines the final prize winners. I submitted this file exactly one time. It was processed and scored by Kaggle and I followed the directions exactly as the competition admin stated. I did not check a checkbox because I did not believe it applied - I only submitted a single file and I recieved the 'all zeros' response from Kaggle's system as it said it would.
Also, the text Anthony quoted is not applicable to the Flight Quest competition either. This is the text that is used for the single phase competitions, not a two phase competition like Flight Quest.
If you haven't competed before I know the difference might not be clear. But if you read the response from the other competitors in the forum, you can see there is general agreement that the submission page is faulty for a two phase competition.
The 23 entries that were submitted are for the public leaderboard set. The purpose of this set is so over the duration of the competition, competitors can see how they are performing relative to each other. These 23 submissions have no impact on the final prize winners.
A different data set, called the final data set is what actually determines the final prize winners. I submitted this file exactly one time. It was processed and scored by Kaggle and I followed the directions exactly as the competition admin stated. I did not check a checkbox because I did not believe it applied - I only submitted a single file and I recieved the 'all zeros' response from Kaggle's system as it said it would.
Also, the text Anthony quoted is not applicable to the Flight Quest competition either. This is the text that is used for the single phase competitions, not a two phase competition like Flight Quest.
If you haven't competed before I know the difference might not be clear. But if you read the response from the other competitors in the forum, you can see there is general agreement that the submission page is faulty for a two phase competition.
Thanks for your reply and clarification.
1. So your stance is that Anthony, the CEO and Founder of Kaggle, is mistaken about the rules and execution of the 2 phase competition that his company is running, even after 8 days of his company communicating back and forth in email correspondence with you?
2. In the forum you initially state:
"As I have been saying over email repeatedly over the past 8 days, and yesterday on the phone with you and the Sorkin, there is no checkbox to check!"
..then later state:
"What on earth is a checkbox doing on the submission screen of a two phase competition? There is no ambiguity - there is only a single file type that is valid for the final leaderboard file. Why on earth is your software not smart enough to figure that out?"
and later:
"I submitted exactly one final submission. At this point the checkboxes should not matter"
and in your reply above state:
"I did not check a checkbox because I did not believe it applied"
While I'm not familiar with all of the intricacies of the competition, conflicting statements like those have me a bit confused over this whole thing.
3. $30k is a lot of money, so I sympathize with you for missing out on that given that your algorithm is better than what the final results suggest. That being said, the other entrants that won competed using the same submission process that you did, was it just bad luck that your entry was not accepted in the manner you had expected or why did they not also encounter this submission bug?
4. Is this your first two phase competition with Kaggle? Was this competition executed different than other competitions? I ask as you seem surprised/confused about this "checkbox" that appears to be at the center of the controversy.
1. So your stance is that Anthony, the CEO and Founder of Kaggle, is mistaken about the rules and execution of the 2 phase competition that his company is running, even after 8 days of his company communicating back and forth in email correspondence with you?
2. In the forum you initially state:
"As I have been saying over email repeatedly over the past 8 days, and yesterday on the phone with you and the Sorkin, there is no checkbox to check!"
..then later state:
"What on earth is a checkbox doing on the submission screen of a two phase competition? There is no ambiguity - there is only a single file type that is valid for the final leaderboard file. Why on earth is your software not smart enough to figure that out?"
and later:
"I submitted exactly one final submission. At this point the checkboxes should not matter"
and in your reply above state:
"I did not check a checkbox because I did not believe it applied"
While I'm not familiar with all of the intricacies of the competition, conflicting statements like those have me a bit confused over this whole thing.
3. $30k is a lot of money, so I sympathize with you for missing out on that given that your algorithm is better than what the final results suggest. That being said, the other entrants that won competed using the same submission process that you did, was it just bad luck that your entry was not accepted in the manner you had expected or why did they not also encounter this submission bug?
4. Is this your first two phase competition with Kaggle? Was this competition executed different than other competitions? I ask as you seem surprised/confused about this "checkbox" that appears to be at the center of the controversy.
Thank you for reading the forum trying to better understand my POV.
re: #1. Yes, my stance with Anthony is that he doesn't think there is a bug in the software because he thinks this is a typical one phase competition. Flight Quest was more complicated than the typical Kaggle competition.
re: #2. I have done a poor job explaining the checkbox. I understand the confusion, as my statements are contradictory. In a single phase competition, the checkbox is important because over the course of the competition, numerous submissions are scored and you ultimately need to pick the one you want to apply as your final submission. In the case you do not make a final selection, the system selects the most recent submission.
The same UI is used for the two phase competition, including the same text that Anthony posted in his response to me, however the submissions work slightly differently as I mentioned.
> "As I have been saying over email repeatedly over the past 8 days, and yesterday on the phone with you and the Sorkin, there is no checkbox to check!"
I said this because I don't believe there should have been a checkbox on the screen. That is what I meant by saying there is no checkbox to check.
re: #1. Yes, my stance with Anthony is that he doesn't think there is a bug in the software because he thinks this is a typical one phase competition. Flight Quest was more complicated than the typical Kaggle competition.
re: #2. I have done a poor job explaining the checkbox. I understand the confusion, as my statements are contradictory. In a single phase competition, the checkbox is important because over the course of the competition, numerous submissions are scored and you ultimately need to pick the one you want to apply as your final submission. In the case you do not make a final selection, the system selects the most recent submission.
The same UI is used for the two phase competition, including the same text that Anthony posted in his response to me, however the submissions work slightly differently as I mentioned.
> "As I have been saying over email repeatedly over the past 8 days, and yesterday on the phone with you and the Sorkin, there is no checkbox to check!"
I said this because I don't believe there should have been a checkbox on the screen. That is what I meant by saying there is no checkbox to check.
IANAL but here's what I think an attorney who took the time to look the whole thing over would say:
The language in the contract (that you posted) seems pretty cut and dried, but I don't know whether that's actually going to hold up in a court of law. You entered a contest expecting the contest holders to hold the contest correctly. When the flaw was exposed, they did not honor it. That means your work was in vain. That's not in the spirit of the contest, thus the written words are only part of the story. A reasonable person expects that the company running the contest does so fairly. You seem to have not been treated fairly thus I think you have a case. I would imagine that, if you posted up a $500,000 lawsuit with $30,000 of reward and $470,000 of punitive damages, you'd hear from a different GE exec who has a very different POV.
The language in the contract (that you posted) seems pretty cut and dried, but I don't know whether that's actually going to hold up in a court of law. You entered a contest expecting the contest holders to hold the contest correctly. When the flaw was exposed, they did not honor it. That means your work was in vain. That's not in the spirit of the contest, thus the written words are only part of the story. A reasonable person expects that the company running the contest does so fairly. You seem to have not been treated fairly thus I think you have a case. I would imagine that, if you posted up a $500,000 lawsuit with $30,000 of reward and $470,000 of punitive damages, you'd hear from a different GE exec who has a very different POV.
In the UK at least there are quite stringent rules on how you can run a competition - give away a car? Abide by the rules.
I did not realize this might be a gray area.
I will do some research on how I can find some representation. Any tips, or experience with who I might get in contact with? Am I looking for some sort of technology or competition specialist?
I will do some research on how I can find some representation. Any tips, or experience with who I might get in contact with? Am I looking for some sort of technology or competition specialist?
http://www.socialmediaexaminer.com/social-media-promotions-a...
I suggest you google for lawyer sweepstake New York
And lawyer contest law New York - talk to a couple till you get a feel for it. It about exploring your options not going to court tomorrow, so the gung-ho ones might not be best. But that's a Brit talking :-)
I suggest you google for lawyer sweepstake New York
And lawyer contest law New York - talk to a couple till you get a feel for it. It about exploring your options not going to court tomorrow, so the gung-ho ones might not be best. But that's a Brit talking :-)
Sounds like they tried to mitigate the situation in a reasonable manner. If they name you as fourth, then the formerly fourth place finisher gets upset.
Some things are like futbol - there's no instant replay. Lawyering up isn't going to impress anyone.
Some things are like futbol - there's no instant replay. Lawyering up isn't going to impress anyone.
I get the instant replay analogy, but I have a problem with a bug in their software impacting the results of the competition. I work in software, so I get living with bugs and understanding they are always going to be there. But I am unhappy that Kaggle/GE announced the winners via press release at the same the rest of the competition field was notified. It makes it impossible for a competitor to dispute the final results.
From the response I have received from Kaggle/GE I get the impression, under no circumstances what so ever would they go back and change the published results of this competition.
From the response I have received from Kaggle/GE I get the impression, under no circumstances what so ever would they go back and change the published results of this competition.
>"It makes it impossible for a competitor to dispute the final results."
Yes. That's the purpose.
The teleology of gamified competition is toward declaring a winner. The rules are structured with that in mind. Those rules are different from the marketplace where protesting and litigation are allowed and consequently, declared winners are simply a matter of opinion, not fact.
Accepting mistakes as part of the game is sportsmanship. And it was a game. Best to move on to the next one: http://www.kaggle.com/competitions
Yes. That's the purpose.
The teleology of gamified competition is toward declaring a winner. The rules are structured with that in mind. Those rules are different from the marketplace where protesting and litigation are allowed and consequently, declared winners are simply a matter of opinion, not fact.
Accepting mistakes as part of the game is sportsmanship. And it was a game. Best to move on to the next one: http://www.kaggle.com/competitions
>'... declared winners are simply a matter of opinion, not fact'
I certainly agree with that part of your statement.
I have a hard time thinking about moving on to a new competition, this one has left such a bad taste in my mouth. Maybe time will change my mind, but certainly not happening any time soon.
I certainly agree with that part of your statement.
I have a hard time thinking about moving on to a new competition, this one has left such a bad taste in my mouth. Maybe time will change my mind, but certainly not happening any time soon.
I understand where you are coming from.
A bit of my background - I referee futbol (or as we call it, soccer). Every game, I try to give my best performance to the players. Sometimes, I make bad calls. The score is still the score.
Obviously, you're pretty damn good. Your competitors deserve to compete against you. Cheating is unfair. Mistakes aren't. Play long enough and it all evens out. At least statistically.
Good luck.
A bit of my background - I referee futbol (or as we call it, soccer). Every game, I try to give my best performance to the players. Sometimes, I make bad calls. The score is still the score.
Obviously, you're pretty damn good. Your competitors deserve to compete against you. Cheating is unfair. Mistakes aren't. Play long enough and it all evens out. At least statistically.
Good luck.
If you are interested in working with/through Kaggle on a longer-term basis, finding some solution outside of a lawsuit would be something to think about. You are getting a chance to interact with them and maybe build contacts that could be useful down the road. If the offer was insulting and/or you don't want to work with them down the road, that is another matter. When you start bringing in lawyers on both sides it becomes an entirely different matter than what it "right". It becomes a costly legalese/terms/courtroom issue. Good luck either way.
Ironically, that has been the best part about this ordeal - a number of the top competitors have shown support through the forum as well as private emails. I compete alone, so this issue has given me the opportunity to get to know some of the other folks.
I am not certain I want to work on anymore Kaggle competitions. I put (just a guess) somewhere between 100 and 200 hours into FlightQuest. It is hard for me to rationalize putting this amount of effort into another Kaggle hosted competition after going through the experiences I went through for this competition.
I am not certain I want to work on anymore Kaggle competitions. I put (just a guess) somewhere between 100 and 200 hours into FlightQuest. It is hard for me to rationalize putting this amount of effort into another Kaggle hosted competition after going through the experiences I went through for this competition.
I notified Kaggle.com within ~1hr of their announcement of the mistake and they did not get back with me for 2 days. 8 days after the announcement was made I was finally able to get a call with a Kaggle executive. The executive offered me partial prize money and kaggle competition points to resolve the matter unofficially. I said I wasn't interested. Kaggle then opened a new forum discussion [1] which a number of other competitors chime in and explain the limitations/bugs of the Kaggle.com platform specifically as it related to the FlightQuest competition and discuss my specific situation.
Finally, an executive from GE contacted me this morning over email and let me know that he supports Kaggle.com's decision of not recognizing me as the 4th place finisher.
I feel like Kaggle/GE have not handled this properly and I would like to get some opinions on what, if anything I should do.
[1] https://www.gequest.com/c/flight/forums/t/4284/acknowledging-two-more-great-competitors