For me there is one thing that works so much better than anything else: new goals. To the end of my PhD, that meant to just suckt it up and finish the work despite low motivation and frankly also workweeks far below 40h.
Later in life, motivation came naturally, I worked more and I am really happy with the outcomes. Sorry, that this is no advice that is immediately helpful, but I hope that it can be reassuring that streaks of low motivation as a student are kind of "normal" and not necessarily something that needs complex solutions.
If I think of myself as a user, that is spot on for information/entertainment, but far from the truth for products. The most extreme examples being everyone using "$THING Wikipedia" but nobody I know using "$THING Amazon" in Google search
How exactly could this be done reliably? With shorts you have to commit to a timeframe for not only finding evidence to reject the claim but also for the news to break and convince other investors.
Meanwhile you can be targetted by financial actors that play your positions on the stock market, at times even disregarding the atual company behind it. Further you have to mind monetary policy. If faith in the stock suffers compared to other stock but the overall market just goes up and up, your shorts have a problem again.
Personally, I have no informed opinion on EmDrive, but I have been very pesimistic about companies in the past and I am about some right now. Still, I do not think that shorts are anything appropirate for me. I'd much rather buy competitior's stock than burn myself with shorts or waste hundreds of hours getting into the financial market that I could also use to create value in my actual job/profession
There may be a lot of uncerntainty in Data Science and ML projects. However, recently I started feeling like I actually have it better than someone from pure software engineering sides of things:
For either, there is often a function from time spent to quality. 100% perfection is basically impossible and before that the function increases very slowly, seemlingly logarithmicly.
For SWE, expectations are often close perfect solutions. Too greedy effort estimations cause a lot of trouble. For DS/ML, however, perfect is usually off the table and this fact is widely (not universally though) accepted. When it is accepted to give estimates in this way, suddenly there no harm from being quoted on it and I really don't mind to give estimates anymore, where I just make a guess at a good 80/20 point. If I am wrong with that point, chances are nobody on the outside/higher up ever knows.
This may be different in domains where very clear targets have to be met (e.g., "self driving cars that pass lawmaker's requirements for use on the streets") and then I'd guess it is a true nightmare.
Like this, I never felt overly pressured by ML/DS deadlines over the last years. Some things were great successes, sometimes the quality wasn't great enough and projects were stopped or customers left. But there never really was a case where anyone thought that working extra long might have been an option to meet higher expectations.
I don't really have a solution for SWE, I don't really see how one would sell something like "I can do it in X time and it will only crash / stop working / make mistakes / become too slow / have vulnerabilities so often. More time will lead to fewer problems". This just isn't what's expected. But at least for complex systems and security vulnerabilities, I'd argue it is actually quite true. Guarantees for 100% perfection just aren't realistic. Avoiding the most obvious pitfalls is done rather quickly and the more time spend, the more is needed for further improvements.
I really wonder about the charging infrastrucuture. I guess it is doable and a necessary transmission, but I am a bit afraid it may be the next thing some countries are sleeping on.
As a German, there are serious subsidies for home owners to install one right now. However, I just moved into a new rental apparment and visiteted quite a few places that were all built in 2020. All of them had very nice parking spaces allocaed to the flat, but zero wallboxes for the entire appartment. I also looked into buying a flat and often it would have been difficult, sometimes even impossible to install one on my own behalf wihtout checking with all other buyers (and these kinds of changes often lead to tedious legal fights, afaik). The place I'm moving to doesn't have one either, but the ladlord will install one, once needed. At the moment I still have a car with a diesel engine and no plans to change soon(I go almost everywhere by bike, even have a different one for rainly days and to carry groceries, and do 0-2 longer trips per month and ~1 very long trip for vacation per year, bike + diesel seems to fit that quite well) soon, but the next car will be electric i guess
To make things worse: The overall power comsumption should not be too much of a problem, but if almost vehicle was electric and charged where people live, the power infrastructure could be in serious trouble. If improving it in remote areas goes anywhere as well as FTTC/FTTH internet, we're headed for disaster. There are a lot of interesting ideas, e.g. decentralized batteries within people's homes and renewables. But if all the focus is on changing the cars on the road, I have little hope that other transitions will be quick enough
Zalando seems really nice. I have no connections at all, but they really won me as a loyal customer because I really prefer the experience over all alternatives (especially including Amazon and local stores for everything except really pricy clothing).
More importantly though, I am really impressed and surprised by some of the research coming out of Zalando. Flair[1] has some great ideas (especially at the time of release, when there weren't pretrained huggingface models for everything to build upon) and a really well-written paper. Colleagues also have had a good experience using the software and achieved very results with adapted NED tasks.
Mostly doing several kinds of NLP:
My actual setup is a Windows Laptop to SSH into Linux machines w/ tmux session. However, I really appreciate WSL for working offline, etc.
My main reason: It is the most convenient way to have Unix tools (grep/sort/cut/sed/less/...) and bash available. Cygwin always was a pain, MinGW / GitBash felt much better, but ultimately WSL just feels best.
These tools are incredibly valuable to my workflow. Sure, stuff like pandas can be nice for small datasets, and some data sits in some DB/Kafka/distributed system. But there have been countless cases where unix tools allowed me to take xxGB zpfiles of text and do basic examination or even build baseline models within a few hours.
Sure, there always are alternatives to use these tools and there are many equivalents. But I would always prefer WSL + conda for Linux to a typical "Windows Conda" installation with that weird GUI and the need to install so many different applications to even just look into the first or last few lines of a huge textfile.
EDIT: That said, of course I can/could always just run a juypter notebook under windows using windows cuda + GPU and share files with a WSL bash where I do my modifications. But again, everything within the same systems just feels better (ipython shell magic, no worries about if paths to the same file are really identical, etc) and while this is by no means a game-changer, it is just nicer that way.
While there may be several things missing for many productive use cases (especially inserts/updates), I think QLever (https://github.com/ad-freiburg/QLever) fits that description very well. There's also a public endpoint linked there.
We're actually using fastai in production and will happily switch to v2. Sure, there are serious questions about long-term stability and we know these projects will be high maintenance.
However, they would be anyway: Core models and algorithms are quickly outdated and any change that allows us to achieve similar or better results with less effort in creating training data is easily worth the engineering work.
That said, I really hope v2 feels a bit more like other libraries: extending v1 models has been pretty painful in several occasions. E.g. making some changes to the underlying pytorch models was very straightforward but still using all the goodies for training build into fastai (in particular all the stuff based on the work of Leslie Smith, tuned for best practices inside the fastai universe) was pretty painful. It is awesome to have a library actually implement best practices from latest research, but sometimes all this greatness was pretty hard for me to transfer to changed models.
That said, it has worked for us in v1 and the benefits outweighed the problems by far.
We have had "good" (maybe not BERT/XLNet'ish levels of quality) results using ULMFit. I.e., on almost all problems we got better results than our previous best approaches (mostly LSTM/CNN and self-attention à la https://www.cs.cmu.edu/~./hovy/papers/16HLT-hierarchical-att...).
Thus, we've seen real value out of transfer learning that doesn't require overly much compute power (and actually could even be run on free colab instances, I think).
That said, I agree that the problem is still very far from being "solved". In particular, I have a fear that most recent advances might be tracked back to gigantic models memorizing things (instead of doing something that could at least vagely be seen as sort of understanding text) to slightly improve GLUE scores.
Still, I am highly optimistic about transfer learning for NLP in general
Imho it has gone further. In a way, all the things described as not actually AI now "are" AI, because the term AI has been used in that way so many times.
I don't think we'll ever use a better (more accurate) term for the ML- and data-driven value current systems create. Instead "true" AI will get a new fancy name to build the next hype around in several year.
> The first company to figure out how to leverage the massive unlabeled data that is available will win the AI arms race.
I would expect that this will remain a per-task question for a very long time, maybe even forever. For example, I was amazed at how elegant https://openai.com/blog/unsupervised-sentiment-neuron/ is for an unsupervised model for sentiment analysis.
I think there are numerous similar application in text understanding and I would not be suprised to eventually seen many sota systems learn in an unsupervised fashion.
Still, there remain many other tasks (in general, but also for text understanding) where this and similar strategies do not seem applicable to me -- or at least where I cannot directly imagine how they should be.
I would not be suprised if we end up with many, many successful strategies to apply unsupervised learning -- many completely different, and thousands of problems still being used with the help of supervised learning. Not with a "company to figure out" a general case
Serious question by someone who's not into this kind of stats too much:
What would happen if you took a big set of Facebook profiles and train some (the same if you wanna) CNN to classify picture->f for each f in profile features. Sure, for some features, your model should be able to deliver decent precision.
Does this mean that you quickly found out what features can be predicted from pictures & how well your CNN performs on that? Or is it possible that you just train models from picture->X where X is basically meaningless but significantly correlated with some feature because of the effect portrait in xkcd's "Significant" (Scientists investigate!) [1]
Imho, it's more typical of machine learning. Sure, both can be very similar and are often signs of a lack of deeper understanding.
I'm no ML expert by any means, but I've seen several bachelor/master thesis and even ML competitions where ensembles performed best. Sure, this isn't necessarily aimless stirring and could combine models that really capture different aspects of the data. But often enough it's just several algorithms that do the same general thing, combined to achieve a slightly higher score.
Imho this is most relevant when competitions provide data that is not readable by humans (e.g. simplified: "classify these documents where all words are given as word IDs and never as actual strings").
To me this has a touch of pouring in data, stirring (build many classifiers and plug them together in an ensemble), and getting answers on the right side.
Optimizing hyper parameters goes in a similar direction, imho. I can really see an analogy to stirring
As a user that probably contributes a lot to the trends observed in the article, I think I can explain my user-behavior pretty well.
I really need about ~5 apps on my phone (browser, mail, alarm, somehow whatsapp managed to become one of those, and maybe few others I don't recall now).
All other apps I download for a very specific use case:
- Rent a bike a foreign city that I'll leave again pretty soon
- Navigate during that one trip where I don't expect to be connected to the internet
- Play a game during the single train ride where my Kindle has no more battery
- Check scores during that one playday of soccer where I have no access to my usual channels (TV, Friends, Browser)
- etc
At the moment of downloading I often know for how long I will be using that app. Most of the time it is only on that single day, otherwise for the duration of a trip.
I'm not sure there's too much App makers could do to keep me as a user (aside from sending me a brand-new phone). I don't really like my phone and I really prefer using my laptop or even desktop, whenever possible.
Yes, that was what i was looking for, thanks.
I was interested in details on the evaluation and why the Stanford parser is still widely accepted as state-of-art, especially by non-NLP researchers that want to try some NLP features to work with. By the way, what is a typical use-case for unlabled parses?
as someone who does very, very little on the side of frontends, i think your time is still invested into a transferable skill that i miss out on:
(frontend) framework design, paradigms and trends. If the stable point you mention will eventually be reached, I am sure everything on the way has contributed and those who have been involved in one way or the other will contribute the most to it and will be the ones that can best contribute to iterative improvements on that stable point.
So which hit should google show for a query like "the fox lyrics"? Of course, it should be a page that contains the lyrics, but which one? The most popular one? The one shown by google will be the most popular one.
From an IR perspective, search for lyrics is somewhat strange. I don't think it's easy to say if some hit is more relevant than the other. Sure, more popular sites may offer better UX (or simply less annoying ads), but at the end of the day it's a search for facts.
I wouldn't be surprised if google will serve those lyrics themselves some day. Freebase (and hence the google knowledge graph) already contains tons and tons of information on pratically any popular song (organized in a confusion way via recordings, canonical versions and whatnot) - artist, genre, album, release dates, length of a recording, etc. I don't think adding the lyrics as well is totally out of question. After all multible KBs are user curated and users provides lyrics.
Just like they serve "fact-hits" from their knowledge graph for queries like "movies by spielberg", google should then be able to serve the lyrics directly. While this is bad for lyrics sites, imho it is convenient for users and follows the same prinicple like "movies by spielberg" or "wife tom cruise"
I'm currently a C++ programmer that does not intend to change to go because I want to have exquisit control.
However, this is not because of my views or something like that but because I'm only a C++ programmer at the moment, because I need that kind of control for my current work. If I want to publish a paper or argue on specifics of algorithms and data structues, e.g., like cache efficiency, it is highly convenient to know as much as possible about what your code does in detail and to have the opportunity to force a particular behavior.
The next time I'll be asked to get a complex problem solved somewhat efficiently (I did write a lot of java code before starting to dive into C++ 3 years ago), I might look into go. But as long as the specifics of my solution are "the job", I can still focus on getting the job done but choose C++ over Go.
New C++ features are always nice, because I don't have to use them all and can cherry-pick whatever is more convenient to achieve what I want
Later in life, motivation came naturally, I worked more and I am really happy with the outcomes. Sorry, that this is no advice that is immediately helpful, but I hope that it can be reassuring that streaks of low motivation as a student are kind of "normal" and not necessarily something that needs complex solutions.