Once again, a moment of gratitude for the San Francisco Chronicle. In a time when local news is mostly gutted, I'm grateful to live in the rare mid-size city that has a robust local paper. Real investigative reporting, a serious local political beat, and features that win Pulitzer prizes. Plus a great sports section and restaurant critics!
It is incredible how far the overton window has moved on this issue.
When I graduated in 2007, it was common for tech companies to refuse to let their systems be used for war, and it was an ordinary thing when some of my graduating classmates refused to work at companies that did let their systems be used for war. Those refusals were on moral grounds.
Now Anthropic wants to have two narrow exceptions, on pragmatic and not moral grounds. To do so, they have to couch it in language clarifying that they would love to support war, actually, except for these two narrow exceptions. And their careful word choice suggests that they are either navigating or expect to navigate significant blowback for asking for two narrow exceptions.
It was indeed originally conceived as 5 seasons, but the creator Tony Gilroy has consistently said shortening it was his decision because the production was too long and taxing:
"We were halfway through shooting season 1, coming through Covid, and the monumental size of the show, the effort, and everything else was just dawning on us. We realized that I didn't have enough calories to do it, and Diego's face couldn't take the timing, because it just takes too long to make it."
"By that point, the work that was required to make the show, at its minimum, was just dazzlingly blinding to look at. And Diego was like ‘Oh my god, we told them we’d do five years.’ Nobody, if we were gonna do it like this, you couldn’t physically do it. It was just impossible."
Am I the only one who feels like this is obviated by Docker?
uv is a clear improvement over pip and venv, for sure.
But I do everything in dev containers these days. Very few things get to install on my laptop itself outside a container. I've gotten so used to this that tools that uninstall/install packages on my box on the fly give me the heebie-jeebies.
Yeah, the hack you described for CIs is typical of "80% charting". We have a list of probably thousands of longtail visualization requests and we're way past the 80/20 point.
These days customers who want to go 100% use the Python/R editors and do their custom visualization there. So you do your SQL query like usual, but then pipe it to Python/R for the visualization. Have you tried that, and has it worked for you? Or do you prefer another model?
In a perfect world, how would the charting in Periscope work? How would you want to go from getting 80% very fast, to going to 100%, in the same software?
It's likely that the largest advertisers don't use the self-serve ad manager. They call their rep who operates the system for them. This is why the self-serve ad managers for AdWords, Facebook Ads, etc. are all pretty unloved.
Periscope Data cofounder here. I wouldn't say we're "allergic" necessarily. ;) Like a lot of startups, we're very young and haven't gotten around to building out the website as much as we'd like to.
`jplitz and `ajones are right. As of now we support MySQL, Postgres, Amazon Redshift, Vertica, SQL Server, Oracle, MemSQL, Sybase, Exasol and Google BigQuery. We add more all the time.
(Full disclosure, co-founder of Periscope Data here.)
Periscope Data supports cross-DB joins. We cache the data in our own clusters so queries run really fast, and also so you can do things like join across databases and upload CSVs.
For ping-and-visualize, I'd recommend a tool that'll give you raw access to the underlying data. Some choices are https://amplitude.com/ and https://segment.com/. Happy to chat about pros and cons if you're curious -- [email protected].
I'll just say upfront that I'm the cofounder of Periscope (https://www.periscope.io/) which is specifically marketed at data scientists, so I have a horse in this race. :)
There are two kinds of charts: Charts designed to find information, and charts designed to sell information. The latter are often gorgeous and many-dimensional: Heatmaps, animated bubble charts, charts with time sliders, etc. And by all means, if selling the data is required, then sell it with the best tool for the job.
As for actually investigating the data, it's usually a lot of tables, lines and bars. They're simple to understand, and there's no cleverness in the visualization that might hide critical information.
To answer your questions, at Periscope I've seen:
1. A line graph of amplitude over time. You should see the frequency emerge clear as day. If you want to calculate frequency explicitly, you could overlay a second line with its own axis. Again, super simple, but gives you the answer directly.
2. I've seen a lot of fancy graph visualizations, but nothing that makes me happy. Depending on what you want to know about your graph, maybe a simple table with a structure like:
[node name][node name][weight]
Or:
[timestamp][node name][node name][weight]
A pivot table on top of this data, transoforming the second node column into the table's horizontal axis, can also be useful.
3. OK, obviously I think Periscope is a great choice here. Loads of data analysts use it to visualize time series data on many tens/hundreds of billions of data points.
That said, other good choices are: Excel, R/Stata/Matlab, gnuplot, Apache Pig. And for the data storage itself, IMO Amazon Redshift is unparalleled.
There needs to be a route from the internet to your DB, yes. Sometimes this works because the DB is in the cloud, or sometimes a port or SSH tunnel is opened into a local network.
We plan to treat the email addresses as people who might be interested in our product, and at some point we'll reach out to everyone who signed up and ask if they'd like to try it out.
If you know you don't want to try Periscope, but would like to read the eBook, by all means use the link above. ;)
previously cofounder of periscope data