This is neat -- how would you think about evaluating the quality loss as you change to more efficient models? I saw you did an analysis on the number of messages, but wondering if there's more robust methods?
It’s common journalistic practice. That’s one of the benefits of reporting in mainstream media, they have pretty high standards for what gets published.
That doesn’t mean they’re always right or there aren’t bad journalists, but if you spend time listening to most journalists explain their reporting they often explain exactly how they verify their stories, what they went through to get sources, and what they were and weren’t able to report.
Journalism is one of the few ways we have in a democracy to hold power to account. What I don’t get is what a lack of faith in this institution does for us except give those in power even more ways to be unaccountable for what they do.
Iranians are willing to die (and are dying) to fight the current government. Inflation and the economy has destroyed their hope of a future for decades. Talk to any Iranian now and they’re all hoping for the end of the regime. It can’t get much worse.
Iran is not Syria, there’s a flourishing academic population, the fundamentalists are the ones in power, plenty of people still remember what life was like pre-1979.
I think you’re confusing giving the company a chance to respond with made-up news. The time a news outlet gives for a response is usually proportional to the severity of what they’re announcing.
Giving a company a chance to respond to some hiring numbers when they likely have multiple anonymous sources who have been credible before is a different bar than asking for comment on something that alleges fraud or illegal activity.
I’m really comparing it to my previous ICE cars, both an Audi sedans and a Mercedes. All built in the last 10-15 years.
100,000 miles for spark plugs is extremely rare. I just pulled up a 2022 A4 and it’s 30k. Transmission fluid is 40k miles. I don’t think it’s a stretch to say that the heat, vibration and weight from an ICE engine wears down parts faster. Add in complexity from turbochargers and there’s even more parts that can fail.
The coolant system on an electric vehicle is extremely simple. There’s no radiator even. An ICE coolant system is far bigger and more complex.
Forgetting the home vs fast-charger nuance here, important to remember than electric vehicles have no spark plugs, no muffler, no engine oil, and a very simple coolant system.
My Volvo XC40 Recharge has a 2-year/20,000 mile maintenance cycle, the only work it needs is a check of the coolant and brake fluids and replacing cabin air filters.
Having suffered through broken engine mounts, torn exhaust flex pipes, painful spark plug replacements, coolant leaks, pump failures, and countless other issues with ICE engines, the simplicity of electric is a breath of fresh air.
Of course there’s the question of what happens to the battery over the long term but i’ll take that trade over what ai had before any day.
I think that’s still the outstanding question. Tools like Mode and Hex have already started supporting dbt’s metrics layer, but will big players like Looker and Tableau also move to support it? That’s harder to say
That’s right, they’re solving the same problem, although metric/semantic layers have traditionally lived within BI tools. Looker’s LookML is a great example of a metric layer.
There you define the relationships between tables, which dimensions to expose to your users, and how measures are to be aggregated along which dimensions and at what roll ups. Filters, formatting, and other semantic information rounds out what’s possible, making it easier to build complex reporting without having to pre-build large joined tables in the warehouse.
In theory a semantic layer decoupled from BI can be useful too, as downstream systems can all query a metric. Maybe an operational workflow needs access to revenue numbers or a spreadsheet wants a feed of some fresh aggregates.
The real question is will BI tools adopt a semantic layer they don’t own, giving up control over a key part of their input into their visualizations? Time will tell.
Believe the replacement was because bash 4 was GPL3 licensed, and zsh was not. It wasn’t so much for user benefits than it was for security and licensing.
It starts there, but then once you get into complex workflows that merge data across your product and CRMs it all moves to the warehouse first. Typical flow is a Fivetran or Stitch into the warehouse, lots of dbt models, then business models fit for consumption down stream.
Once in the warehouse, it needs to get back into those operational systems again, which is the tricky part.
I’ve done these one off integrations from the warehouse into Salesforce (creating leads, converting them, moving stages all based off product usage), and into marketing tools (customer segmentation built using SQL in the warehouse, then sent to marketing automation tools).
Being able to feed tools directly off the warehouse instead of writing one off integrations is the real value.
Plans are useless but indispensable. They are definitely worth doing, they give your manager and ultimately the rest of the company insight on product development. This could inform marketing, sales, customer success. Is that feature likely to come next week? Should we start writing a blog post now or can it wait? Are we fixing a major issue for an important customer?
Part of your job as a professional engineer is to both communicate and manage uncertainty. A properly scoped story shouldn’t be a month long project. It could be a few hours, or it could be a few days.
Helpful advice is to give your estimates using an appropriate dimension of time that reflects the uncertainty. If you think it’s going to take 2-3 hours, call it a half day.
If you think it’ll be 8 business days call it about two weeks.
Sometimes you don’t have an estimate. That’s where scoping comes in. It’s perfectly acceptable to say I’m not sure but if I spend the next 30 mins scoping it out, I can get back to you with a better idea.
I find managers are perfectly understanding that the world is a fuzzy place and no one has all the answers. Get comfortable with admitting that to yourself and to others and estimating gets a lot easier.
I think a hard part with lots of these “what do I use x for” examples is it starts with the tool and the discusses the problem that it solves. I find it more helpful to start with a problem, and discuss the various tools that address it, in different ways.
Forget email, say you have a app that scans links in comments for maliciousness. You rely on an internal api for checking against a known blacklist, which follows shortened links first, and an external api from a third party. You want the comment to appear to submit instantly to the poster but are comfortable waiting for it to appear for everyone else. What are your options?
You could certainly use message queues and workers. If you’re cloud native maybe you leverage lambdas. Maybe you spin up an independent service that does the processing and inserting into the database in the background, and all you need to do is send a simple HTTP request on an internal network.
Your solution depends on your throughout requirements, the size of your team and their engineering capabilities, what existing solutions you have in place. Everything has its pros and cons. Pretending that celery/redis is useless and would be solved if everyone just used Java ignores the fact that celery and redis are widely popular and drive many successful applications and use cases.