Expand? The design constraints seem very dissimilar:
- weight is important, form factor isn't
- any app is either doing 3d rendering or integrated into a windowing system ala a trad desktop
- they are not constantly-on, they are used for long periods with breaks
- they are limited by processing power not by UX
- the peripherals are very complex - to the extent of complex requiring access to hand and/or controller movement at a very fine level
If anything they are similar to laptops, but overall they are their own device class.
If "similar to mobile phones" means "similar chips and batteries"... sure, but so are laptops build after 2020, besides that I don't quite understand the comparison.
> You’re not a mega company
Right, I'm asking why Steam, HTC and Meta did this, which are in the top <x> tech companies by market cap and spend billions on this stuff. Totally get why a small VR device dev would not do this.
> OS, drivers and libraries for (3D) graphics up and running
Right, except that I am not saying one should build a kernel, and webgl, cuda, etc are kernel dependant not OS dependant
> assurance that you still can get hardware to deploy your stuff on X years from now
I am unaware of android-specific hardware features to date (linux specific, maybe, but not android) -- you can literally add any bootloader and load a linux kernel and get drivers running on most phones even, w/o android, and where "android" is necessary it's simply an artifact of the company maintaining the phone having build the drivers into their distro and not released them separately.
> You also don’t want to order a million units up front.
Again, seems unrelated, I'm talking about building the VR hardware i.e. not order off the shelf and white label (unless your claim here is that meta, htc, steam etc are doing white labeling, which doesn't seem to be the case)
Right, I'm saying for a VR devices, no a mobile phone. It makes perfect sense why android would be the choice for a mobile phone. I'm rather confused what you are replying to.
it's more like, in a world with thousands of oncologists that treat pancreatitis cancer, saying: You should go to one, I'm not going to recommended mine in particular, but I went to one and she cured my cancer, so maybe you should give it a shot.
Did add a quick explanation about methods at the end ... I forgot that when I say "you can read the science around this topic to see how" and provide some links to examples of papers, people don't actually do it.
There are at least 6 cloud providers I can name that I've used which run their own data centers with capabilities similar to AWSs core products (ec2, route53, s3, cloud watch, rdb)
Ovh, scaleway, online.net, azure, gcp, aws
That's one's I've used in production, I've heard of a dozen more including big names like HP and IBM, I assume they can match aws for the most part.
...
That being said I agree multi tenant is the way to go for reliability. But I was pointing out that in this case even the simple solution of multi region on one provider was not implemented by those affected.
...
As for running your own data center as a small company. I have done it, buying components building servers and all.
Expenses and ISP issues aside, I can't imagine using in house without at least a few outages a year for anywhere near the price of hiring a DevOps person to build a MT solution for you.
If you think you can you've either never tried doing it OR you are being severely underpaid for your job.
Competent teams to build and run reliable in house infrastructure exist, and they can get you SLA similar to multi region AWS or GC (aka 100% over the last 5 years)... But the price tag has 7 to 8 figures in it.
This seems like an insane stance to have, it's like saying businesses should ship their own stock, using their own drivers, and their in-house made cars and planes and in-house trained pilots.
Heck, why stop at having servers on-site? Cast your own silicon waffers, after all you don't want spectrum exploits.
Because you are worst at it. If a specialist is this bad, and the market is fully open, then it's because the problem is hard.
AWS has fewer outages in one zone alone than the best self-hosted institutions, your facebooks and petagons. In-house servers would lead to an insane amount of outage.
And guess what? AWS (and all other IAAS providers) will beg you to use multiple region because of this. The team/person that has millions of dollars a day staked on a single AWS region is an idiot and could not be entrusted to order a gaming PC from newegg, let alone run an in-house datacenter.
edit: I will add that AWS specifically is meh and I wouldn't use it myself, there's better IASS. But it's insanity to even imagine self-hosted is more reliable than using even the shittiest of IASS providers.
The market needs to open up for companies selling NFTs representing IpV4 address blocks. Not to provide any right to it's usage or anything, just for you know, the bragging rights.
You may not be the target audience though, the target audience are probably people that aren't yet "bough into" the google ecosystem. So they are unaware of this axing policy. Anyone logging into chromium, using gmail and google calendar and an android device is already "theirs".
What they want is to increase the share of people that are in the MS ecosystem (and presumably also OSX ecosystem, but that's mostly a status signaling thing, so the strategy there is probably different).
We are honestly very happy to hear about every usecase people have for this right now. It's users that help us shape the product to a large extent, at the end of the day.
We can chat more about it here or feel free to pick one of the many contact routes scattered around the thread :)
As far as I can tell MADlib is not automl, it just provides various statistic analysis and "classical" ml algorithms as function/macros in various database, and integration seems to be quite different from the way we do it (and I'd say more complex for the user, but maybe that's just my bias talking).
So I don't think there's a lot of overlap there. But if you think otherwise and work on it or know someone that does, I'd be quite excited to have a chat, just to share experiences and tips if nothing else.
If you're interested in participating in a "beta" release of that, we have a newsletter: https://mindsdb.com/newsletter/ where I'm 99% sure it will be announced, and in case you don't get a code ping me and I'll send you one :)
But the timeline for when to release this is still inexact, since ideally I want the "beta" to be as stable as possible, as to not have to get people to migrate to a different version later.
Also, if you have the time to share your usecase in like 3-4 paragraphs please do, either here or email me, because we design this with users in mind at the end of the day.
> So I assume that you are doing hyperparameter search? Can you share what optimization method you are using for search (e.g. random, gp )?
Short answer is optuna and ax but only sometimes.
Long answer lead me down a rabbit whole and it's 10k+ words and a few experiments deep. If you're interested in this are specifically ping me, but I've got nothing concrete, however I like discussing it. A recent paper I saw that somewhat echos my thoughts is: https://arxiv.org/pdf/2102.03034.pdf | but some bits feel either over my head and/or overly pedantic and/or overly formal | and I'm not sure I agree with the conclusion | and loads of it is irrelevant. But if the problem interests you I'd suggest giving it some time, with those disclaimers in mind
> Also, is the search can be distributed in parallel to multi node ?
Theoretically yes, practically it's still WIP to get this to work, but the architecture we have right now is very much conceived with massive distribution in mind (see our docs for more details on that).
> And, if mindsdb is not part of the db, what happen if minddb fail ?
The select query you use to make a prediction returns an error, essentially. Assuming you mean "what happens if it crashes or if the model you are using crashes?".
e.g:
psql> SELECT diagnostic FROM mindsdb.flu_detector WHERE headache=true AND temperature=37.5 AND cough='mild';
psql> Error: External table returned error: "Segfault"
OR
psql> SELECT diagnostic FROM mindsdb.flu_detector WHERE headache=true AND temperature=37.5 AND coughsfsagsa='mild';
Not at the moment, but we're going to add it very soon, with the first implementation allowing retraining with a certain user-set frequency (e.g. once every 2 hours).
Which will allow the model to be always fresh as new data comes in (assuming there's no time limit on the query)
Looks quite interesting, already pinned this in the relevant slack channel :)
To be honest I'm rather happy with how the internal benchmark suite is turning out, but to some extent you are inviting bias by creating them yourself. On top of that, it doesn't hurt to have more benchmarks.
At the end of the day it's a combination of:
* How much work is it to integrate (easy to measure)
* How visible is it, i.e if we actually find something interesting will be visible and legible to others (ify to mesure, citations, stars, etc are some invitation)
* How useful it is to "improve" the library (hard to measure, and what we aim to be good at is a moving target)
So realistically that's the equation I have to judge in terms of adding a new benchmarks suite, and it's very annoying because you'll note the most important things are the hardest to measure.
Would you want people to integrate with this now or would you rather wait a few weeks/months/years until it matures more? If the former, can you give a few details regrading where to start (README is fairly barren), if the later please ping me ([email protected]) when you think it could be ready to try.
Anyway, any open benchmark library is a step in the right direction, thanks for working on this :)
From the user perspective it's inside the database, you can run mindsdb in the backgrond,connect it to the database once, and then do everything from within the database (i.e. connecting with a sql client to your database server and issuing commands the same way you would query "normal" tables).
From a technical perspective it's a separate server that communicates with the database through various mechanisms (e.g. federate engine) but it's no different from e.g. multiple instance of mariadb being abstracted by a galera cluster into something that behaves like a single database from a client perspective.
I would love to support Scylla, I ** love that database, those guys are magicians. And I assume in supporting that we'd also offer de-facto support for Cassandra.
I don't think either Scylla or dynamo are on the roadmap now, but if you want them feel free to create an issue asking for them: https://github.com/mindsdb/mindsdb
It should be noted that there's two level of support:
1. As a source of data (easy to implement)
2. Being able to publish models into the database (a bit harder)
If you work with those and are interested in doing ML from the database please get in touch, ideally via github, but you can also use the contact form (https://mindsdb.com/contact-us/) or email one of us directly. The best case scenario for us is that when we do one of these integrations we have an actual user in mind, and we're open to "first users" for any database where we can find a reasonable way of integrating.
I haven't looked into it myself, but I'll try to understand what they do better, thanks for letting us know.
I will say that:
1. It's not open source, so hard for us to compare other than running black-box experiments.
2. Oracle, so presumably that comes with all the Oracle-ecosystem buy-ins that implies, which might not be ideal for many people.
As a purely personal opinion:
I guess it's good to know that other people are thinking in the same direction as us, but at the same time I personally would like for widely-used ML libraries to be open-source. If these models are going to be used as generator of important decision making algorithms, ideally both the model and the algorithm should be open source. The later is up to whoever is building the algorithm, but I think if we can get the zeitgeist to move towards the later being open source as the norm that can alleviate a lot of potential harm and has little downside.
I.e. Do you feel comfortable with the NHS off-sourcing important decision making to algorithms that are proprietary black boxes? Considering that it's funded by the tax paying public and it's supposed to service that public.
"Secret" laws used to be a norm in the past e.g. in large civilziations like the Roman empire, where the norm evolved to be that only "schooled" men could understand the law due to complexity, or in most of medieval Europe where the bible was foundational for morality but closed off to a small subset of the population that knew Greek or Latin and could get their hands on it. But in general that seems to have caused more harm than good.
It seems reasonable to ask that, if algorithms are going to be used by governments in decision making, those should be entirely open. Ideally the ones used by corporations should be open to whatever degree is possible, to avoid run-off harm from buggy or unaligned systems.
If anything they are similar to laptops, but overall they are their own device class.
If "similar to mobile phones" means "similar chips and batteries"... sure, but so are laptops build after 2020, besides that I don't quite understand the comparison.