Takes a book in English, finds the words that are "rare" (by the measure IDF >= 15 or -ln(frequency) > 14) and lists the definition in a dictionary and the places it occurs in the original text. The final output is a static HTML file. You can view a live demo for a subset of words from a book here: https://deedy.github.io/vocabuliwala/
Some examples output demonstrating some of the key features of Vocabuliwala:
- Dictionary lookup
- Secondary dictionary lookup: looking up dictionary meanings which themselves need explanation
- Difficulty rating
- Show appearances in the input book
Who is this for?
- A casual reader of a book
- A teacher trying to teach a book
- Students trying to learn vocabulary for a standardized test
Curious to get the community's feedback and see how I can improve this!
And it's an even bigger problem with MEng/MS students. These are usually unfunded cash cow programs even at top universities. They accept fairly mediocre students from China and India and the class is usually 80% Chinese/Indian. A generalization, of course, but they have 0 intellectual curiosity. They are here to pay $50k-60k for 1 or 2 years, make sure they have as close to a 4.0 and then go get a tech job where they will make $150k/yr, and little to none of their skills from class would be needed.
And I can speak for Indians, but CS education in India aside from the IIT, the IIIT, BITS and some NITs is dismal. Cheating is rampant there, and they're much more well versed with the art because it's much harder to cheat and get away with it in India - you can't bring phones to your exam or freely go to the bathroom mid exam, for example.
I was a TA for a graduate level class at one of the top universities in the US and I've had some interesting encounters with plagiarism.
I. The time I got caught for "plagiarizing".
In an intro systems class, me, a CS major, and my roommate, who wanted to minor in CS, were working together and I was "showing him the ropes". He was an intelligent student and we never worked together on the homeworks aside from general verbal discussions on what the solution could be. He used a Windows laptop and for one of the assignments, his C code wasn't compiling because he was missing some libraries and he told me he couldn't figure it out and we were approaching a deadline and asked me to compile it for him and send him back the binary. I did so, but when sending back the binary, in a rush, I accidentally mistook my HW folder for his (we'd downloaded this as a part of the assignment, and the folder structure was identical) and sent him my binary by mistake. Both of our solutions worked.
Obviously, we got "caught" in the most naive way. Our binaries had the same MD5 hash and the CMS flagged us. We were both confused at first, and then we realized what happened and explained it to the professor.
The proof was simple - just compile my roommate's binary and run it. However, he annulled our assignment to 0. We still both got As (because you could drop one homework) and while some may claim this was a gentle slap on the wrist, it felt unjust. We clearly made a dumb mistake and we shouldn't be punished at all, especially when we knew how rampant actual plagiarism was.
II. The time I caught students for "plagiarizing".
As Kevin points out in his post, there aren't really any incentives to catch students for cheating. As a TA, I get no benefit, and moreover, there's a cost. No one wants to be known as THAT TA who busts kids for using "a little help". Keeping that in mind, I was usually very lenient when it comes to cheating. I've noticed signs, but there was never enough proof to warrant the effort of calling someone out.
However, at one level it went too far. Two students who were partners for the "projects" had submitted nearly identical solutions for a complex Graphics homework assignment. They got the answer right, but I looked into their working and they both said "(9/5) / (4/3) == (4/7) / (5*9) = 1/3". I don't remember the exact values, but it was two steps of non-sense numbers and then a correct answer. I ended up reporting the case, mostly because I felt like my intelligence as a TA had been insulted. Are you seriously going to submit random numbers with a correct solution hoping I won't see? In any case, it didn't go anywhere.
III. Discovering a cheating ring.
At our university, one of my good friends and project partners told me there was an "enormous Asian cheating racket" - not to call out any specific race, I'm Asian too. I wasn't surprised - to be blatant, it made sense. We're very grade oriented with tiger parents. Then I learnt the extent of it. There were apparently Chinese forums and "outsourcers" you could send your homework problems to and they would solve it and give it back. In addition, there were special shared systems like DC++ where you could discover answers to homeworks for different classes at my university as well as Prelims, Midterms and Finals contributed by students of previous years. I was in shock. Students would leave exam halls to go to the bathroom just to look at these answers mid-exam. But was I gonna tattle? No.
IV. The reality at universities.
Not just in CS, but in every other subject, almost everybody cheats. Excuses that go around are:
"I've worked on it with someone else"
"Oh the TA in office hours told everybody the exact same solution"
"What? Cheating? me?"
"Maybe he/she took it from me, I didn't do it"
And look, people aren't stupid. We all know how cheating works. You get a homework assignment, and you re-write the sentences in your own language. You get some code from someone else and you define some useless functions with 1-2 lines of code. Or you arbitrarily re-organize lines of code. You rename all the variables. You re-organize your functions. You create some unnecessary classes.
There were students who distribute 10 homework assignments between 10 people (in groups of 2), and have one do the assignment (use office hours, friends, google, whatever) and the other literally re-write the assignment in LaTeX 9 different ways for the others to use. No one would ever really have to do the work.
The well known key to cheating is plausible deniability - if there's enough evidence you didn't do it, you didn't do it.
Interesting article! The metric you use does eliminate triviality, but it sometimes uses very obscure (and arguably uninteresting words), such as calumnies, ivoriness, coprophagist, etc. That's what you describe as
Webster's Second jargon that nobody knows".
It would be interesting if you could adapt your metric to account for general prevalence of the word in English. Scan a giant subsection of say Wikipedia, and assign a frequency to each of the 234,000 words in a map, giving unseen words an infinitely small frequency, and then use the sum or multiple of the frequencies of each of the anagrams to bring out some truly interesting ones!
Great read. I think the title "On Reddit, the earlier you comment, the louder your voice." or something to that tune would've made for a more impactful headline!
That's a super interesting thought. You should consider that the sum total of popularity of topics on HN up till today can't be used in hindsight as a predictor. It would be interesting to see if we merely looked for past spikes in keywords and used that to govern investment decisions. Even then, I fear that for every "bitcoin" and "apple", there may be other technologies and companies (especially smaller startups) that didn't work out so well, although I hypothesize a net positive.
Despite it being public data, because the information circulated on HN is at the core of technology, it could prove valuable to investors with limited knowledge of it (and might well be worth packaging and selling, haha).
I should've mentioned that all the times are in UTC. I'll work on normalizing them to PST - it's pretty confusing right now. Thanks for letting me know!
SELECT
author,
COUNT(1) AS c
FROM
[fh-bigquery:hackernews.stories]
WHERE
author IS NOT NULL
GROUP BY
1
ORDER BY
2 DESC
LIMIT
1000
and armed with the knowledge that HN has been in existence for 3158 days, there are 11 people who post strictly more than once a day. They are:
1 cwan 7077
2 shawndumas 6602
3 evo_9 5659
4 nickb 4322
5 iProject 4266
6 bootload 4212
7 edw519 3844
8 ColinWright 3766
9 nreece 3724
10 tokenadult 3659
11 Garbage 3538
Just under 1 a day: robg 3121
I'm 100% sure LinkedIn probably offered to buy Coursera at a similar price:
1. A lot of Coursera's engineers are LinkedIn veterans.
2. Coursera is arguably a much better quality and more highly regarded education platform than Lynda.
3. Coursera certifications were something LinkedIn and Coursera were actively working on.
I would imagine that Coursera refused because their vision of education is much broader than what they might have been able to achieve within LinkedIn. What do you guys think?
> It's interesting that one of the only Ivy programs which is essentially immune to this gaming of the system is Harvard Business School
Having known many peers at HBS, I highly doubt the truth of your claim. It seems extremely biased based on your own background. Do you have any evidence?
As stated in the article, I think the hardest barrier for this project will simply be the lack of branding. College has pretty much boiled down to institutional prestige and the credential that comes with it.
Gauging "success" within this system seems hard. It's hard to attribute success to a university when it's prestige ends up determining the calibre of its student body in the first place. And a clear feedback loop exists here. On account of the prestige associated with their college, students end up leading illustrious careers. Their "success" is a combination of their inherent pre-college skill and their credential, one could say.
Now, given the Minerva project, where the feedback loop doesn't exist and the quality of students are towards the higher side, how does one begin to test the efficiency of a new system without any proper control experiment?
Because of this difficulty in assessing the success or failure of this project, even though I hope and believe it works well, the general opinion might be that it failed.
As a Cornell Computer Science undergraduate from India myself, I find this post very thought provoking. Other factors that I personally would probably take into account is gaining some industry experience before one jumps into entrepreneurship in India and simply acquiring some capital from a few years of employment in the US which can you lean back on during your difficult startup days back in India. Just a thought.