Do folks often use Jupyter notebook outside of Databricks?
2 comments
Not sure if this helps, but tons of people in science use local jupyter installations (typically installed through conda). Using externally hosted ones are probably the exception in my circles.
I’ve only used a few hosted ones from time to time, and now I host my own (private) jupyterhub for various reasons.
I’ve only used a few hosted ones from time to time, and now I host my own (private) jupyterhub for various reasons.
So you would rather connect to a databricks lake from a privately hosted jupyterhub if that's an option?
maybe? I'm wasn't even sure what databricks was until I looked it up. I might not be the right person to answer this question, so you might need to narrow down what your audience is :)
My POV comes from academia. Our data is rarely in any sort of database, and is often just in various files we have to parse or otherwise ingest and analyze. So the notebooks tend to live next to the data (on our laptop or maybe a group server).
As an aside, academics often have a very hard time paying for services. Universities want to get involved in any sort of recurring subscription, get the legal department to look at the contracts, maybe even negotiate, etc. This is true even for $10/month. So we often look to do things locally ("shadow IT" is kind of a related phenomenon).
However, researchers and data analysts in private industry probably have a very different way of working, so don't take this as gospel. But as for your original question, yes, many many scientists use jupyter locally.
My POV comes from academia. Our data is rarely in any sort of database, and is often just in various files we have to parse or otherwise ingest and analyze. So the notebooks tend to live next to the data (on our laptop or maybe a group server).
As an aside, academics often have a very hard time paying for services. Universities want to get involved in any sort of recurring subscription, get the legal department to look at the contracts, maybe even negotiate, etc. This is true even for $10/month. So we often look to do things locally ("shadow IT" is kind of a related phenomenon).
However, researchers and data analysts in private industry probably have a very different way of working, so don't take this as gospel. But as for your original question, yes, many many scientists use jupyter locally.
Thanks
Never once used it with Databricks.
What do others think?