In particular, Metabase and Superset can be deployed with DuckDB support. You mentioned customer facing dashboards, note that Metabase embedded is not free. Just to say, our SeekTable also has DuckDB connector (and can be used as an embedded BI).
Websets are cool - I remember that 2 decades ago there was a project in Google Labs that tried to return google search results as 'objects' x 'properties' but it never left their research sandbox (cannot remember project's name unfortunately).
Searches that give tabular results can be cheap if you already have structured datasets (extracted from crawled data), so LLM can simply convert the user's natural language query to SQL query (or SQL-like query) which can be cost-efficiently executed - say, with DuckDB. This approach can also give more correct results - as values in these structured datasets can be validated in the background, not as an individual 'deep research' task.
I understand that this is another kind of search service, however, this can be a way to offer free/cheap searches for users who don't need expensive individual research tasks.
Common approach is to use a data warehouse that is suitable for executing OLAP queries (to load/calculate data for your dashboards) in seconds. In simplest cases ('small data') this can be simply app's database replica - a separate (read-only) DB server that is used for reporting/analytics purposes. It is easy to configure master-slave replication with built-in SQL Server/PostgreSql/Mysql capabilities.
If replica server cannot execute queries fast enough, a specialized (optimized for OLAP-workload) database should be used instead. This can be a cloud service (like BigQuery, Redshift, Snowflake, Motherduck) or self-hosted solutions (ClickHouse, PostgreSql with pg_analytics extension, or even in-process DuckDB). Data sync is performed either with scheduled full-copy (simple, but not suitable for near real-time analytics) or via CDC (see Airbyte).
Just tried gemma3:4b for structured output and it fails with a strange error ( ollama is the latest):
Ollama error: POST predict: Post "http://127.0.0.1:49675/completion": read tcp 127.0.0.1:49677->127.0.0.1:49675: wsarecv: An existing connection was forcibly closed by the remote host.
Not sure this is Ollama or gemma3:4b problem. At the same time, gemma3:12b works fine for the same API request (100% identical, only difference is model id).
This is an extension for standard netcore DI-container. It is possible to have declarative JSON definitions of components (services) in _addition_ to the existing code-based approach.
For example, usage of JSON IoC config may be useful for applications with plugin architecture, or when parts of application are generated from some kind of domain-specific models.
> everything is translated to raw sql then pushed to the database layer
All ROLAP-kind of BI tools do that (including PowerBI when it uses direct-query connection mode), it is expected that underlying data sources are fast enough to handle these aggregate queries very quickly. In fact this approach may be used even with non-OLAP databases (like PostgreSql or SQLServer) and specialized analytical datastore is needed only for really big datasets (BigQuery, Snowflake, ClickHouse etc). In many cases correct usage of report parameters that can filter DB records by indexed columns OR usage of pre-aggregated materialized views, or tuning of SQL query generation (say, avoid JOINs and SQL-calculations when they are not needed for the concrete report) can solve performance issues.
This doesn't mean that Excel's PivotTable (and SSAS cubes) is good and ROLAP-kind pivot tables are bad because their applications are different. In cases when pivot tables should show actual (near real-time) data and this is main purpose of this kind of reports in BI tools; when users need to explore some dataset in a disconnected mode they always may export concrete report's data to Excel - in fact, some BI tools can export their internal pivot table into Excel file with pre-configured PivotTable.
search-driven analytics is not really a new thing and products in this space were before LLM-era. This kind of interface can be useful for some categories of users but it is not a game-changer - prompts cannot replace Excel and its pivot tables, and in fact typing prompts may be even more complicated for users than good old 'clicks'.
You may try these online pivot tables https://www.seektable.com where you can re-order rows/columns simply with a click on the header, apply filters via simple input where you can specify which items to keep or exclude.
PBI can be accessed via XMLA endpoint which can be consumed by many old components that previously were used with SSRS + has 'dataset execute query' REST API, so in this meaning PBI can be used as headless BI. Don't know much about Tableau/Qlik API, do they provide API for querying their internal semantic model?
Solutions that are good for embedding reports have some specifics; say, usually a self-hosted version / white labeling are must-have.
Shameless plug here: take a look at our https://www.seektable.com, especially if you're interested in pivot tables (and affordable pricing, of course!).
It depends on what kind of embedded reports you need. Most BI tools offer iFrame-based integration as simplest way to embed visuals into any kind of front-end.
Our BI tool (https://www.seektable.com) is often used as embedded BI with a very good value-for-money (about $250/mo for on-premise installation with white labeling). It is possible to embed individual reports, dashboards (that can contain custom visuals, your devs will like the approach) and even 'app view' (where users can make ad-hoc queries in a self-service manner).
In 2022 we use NET6, last .NET Core was 3.1 which reaches end-of-life soon.
Our SeekTable started as .NET Core 1.1 app 6 years ago, it was upgraded to 2.1, then 3.1 and now it is a NET6 app. Upgrades up to 3.1 were pain in the ass, API/SDK evolution was quick and not backward compatible, however last upgrade to NET6 was easy - this is more a change of the run-time version, as APIs are not changed anymore and become stable.
Depending on usage scenario, Excel is replaced with:
- process management system, usually web-based, and all business data is stored in the (central) DB. This could be specialized domain-specific solution, or organized with low-code tools like PowerApps, or even own software developed in-house (possibly, with help of outsourcing partner)
- BI tools to replace use-cases when Excel is used for reporting/analytics (not data entering)
Migration can involve both these items, or only one of them. For example, if using Excel for data entering is still ok, but for reporting brings to much chaos (no single source of truth, it is possible to 'edit' or open outdated worksheet and have different numbers etc) - you may start with a BI tool that uses 'master' Excel file as a data source.
> * More than Grafana, I think you need something like Metabase integrated OOTB. That might be a killer feature.
Nowadays you can directly connect to CH from many BI tools, and this could be different choice depending on report types / personal preferences. For example, our SeekTable has built-in connector for ClickHouse, and support 2 different drivers - one for binary TCP interface, another one for HTTP(S) interface.
I'm afraid a long sales cycle (6-12mo) for B2B products that target big companies is a typical case. This remains relevant even if your product doesn't fall into an enterprise software category (in other words, it is low-cost for them).
They can schedule your product evaluation ("integration project") according to their schedule, but very often this doesn't happen at all - so the answer "we'll continue this somewhere in the future" _may_ mean that they don't consider using your product.
You cannot do much in this situation. This lead should be considered 'cold' so it would be enough to ping them from time to time to remind about your product.
It looks like MetricFlow shines in constructing SQL queries on-demand, which means that it should be directly used by a BI tool, am I right with this?.. Generation of the static SQL (with CLI) for each report doesn't seem very usable on practice.
In other words, BI tools needs to have a special connector that automatically utilizes MetricFlow Python API (or CLI). What BI tools already can use MetricFlow in this way (open-source part of the project)?
Actually I'm asking about that as a BI tool vendor. We have added an ability to use custom connectors (web API) so potentially this kind of connector can use MetricFlow for SQL generation.
First of all, I wish you and your family stay safe. Nobody in Ukraine don't want death to _all_ Russians, however we hope all Russians who crossed the border with weapon in hands, will die here. After all, they _always_ have a choice: if a soldier don't want to kill Ukrainians and destroy our cities, he can surrender in Ukraine.
Second, I'm afraid you don't know what "emergently evacuate" means. Completely. Family evacuation is when you're in the car with 2 kids while Russian helicopters attack Gostomel (city near to Kyiv) and go just above your car. Emergency evacuation is how now people try to leave Bucha, Irpin', Mariupol, Kharkiv, Chernihyv, Sumy. This means that their lives are _really_ can end accidentally because of warefare.
Third, my question - if you didn't support Russia government, where you did for the last year? Last month? While Putin collects hordes on the border with Ukraine? Oh, you opened eyes only when West sanctions start working...
I don't blame you. However, all this pain that we feel here in Ukraine, because you (and millions of Russians like you) didn't interested in what _your_ government and _your_ leader prepared for last 10 years. Interesting questions - what is your position about Crimea?.. That was a bright sign that Russia goes to hell, but you was blind.
Microsoft employees with Ukrainian roots, you're the best - I know that you made a lot to make this possible. We'll win together / переможемо разом! Слава Україні!