Score:-1

Server

Is it possible to host the database on one server, but the actual data on another?

Alex Osheter

4/2/23, 9:38 AM

We're planning to do AI research using an HPC. This HPC will use datasets that we've created. These datasets are fairly big subsets of the entire dataset (~1TB). All the data we've gathered from experiments will be stored in an SQL database. We want to use SQL queries to fetch relevant subsets from the database which are relevant at a given time - so for that we've developed a RESTful service, which allows people to send sanitized queries.

There are some limitations that are currently halting our setup.

We have a host for the RESTful service, but using ~1TB storage on it is a bit of a last resort, and we'd prefer to find an alternative way to do things. I was wondering is it possible to host the database on one server, but have the actual data sit on another server? So that when the researcher sends a query to the RESTful service, the SQL server selects which files to send, returns them to the restful service, the restful service returns download links to all the datasets.

We're using MySQL at the moment to store the data, and an instance of Flask to allow researchers to submit new experiments, and fetch them.

0 + 0

mysql

hosting

storage

database

TomTom

4/2/23, 10:53 AM

The question shows a lack of common sense knowledge for system administrators - and the OP has said so clearly. This runs down to product recommendations and teaching basics, both off topic on this site. Superuser.com is more appropriate.

Score:3

Server

Phill W.

4/2/23, 12:10 PM

There are three components here

Flask, which is serving your REST API,
mysqld, which is the running database instance, and
the data files managed by that database instance.

There is no reason why Flask should share a server with the other two and plenty of good, Security reasons why it shouldn't. It will be perfectly happy, given the right ConnectionString to connect to a mysql instance running on another server.
This is probably the best place to "split" your architecture.

The database instance and its data files should be "close" to one another as possible, i.e. with as little as possible to "get in the way" and destabilise your database. (Indeed, I would go further and suggest that you should regard them as a single entity, the database, and forget about "files" completely).

Having a database server with attached disk devices is fine.

0 + 0

Alex Osheter

4/2/23, 6:41 PM

Thank you! I will check if this is possible. To my knowledge, our provider has two tiers - -one is a VM with 1TB of space, and the other is an ESS based solution. If it's possible to run a database instance on the ESS, that's pretty cool. But I don't know if that's possible.

Score:1

Server

TomTom

4/2/23, 10:36 AM

Let me ask you a question - what do you think any SAN ever built is? A storage area network means that the data resides on another machine, purpose built for that. ISCSI is a protocol for exposing block based storage (i.e. virtual discs, not a file share). Why you think it exists?

So, the obvious answer is yes.

0 + 0

Alex Osheter

4/2/23, 10:50 AM

Well I've never heard of SANs, so it's not so obvious. For a scientist, this is not _obvious_. It would be great if you could expand on your answer - what are SANs? How do they work? Is this a solution provided by many hosts? Is it supported by MySQL?

TomTom

4/2/23, 10:52 AM

"For a scientist, this is not obvious" - this is a place, per site rules, ONLY for admins, not for scientists. A system admin not knowing what a SAN is... well... so, the question is off topic here. Also, as Scientist, feel ashamed for not trying to find what a SAN is on google.

Rick James

4/3/23, 5:06 AM

A SAN is a bunch of disks and a computer with nothing to do but shovel data between those disks and a network that has the user (such as the SQL engine) on the other end. There is some slowdown, but it is a great way to get really huge datasets "online". These days, 1TB is quite feasible on even your home laptop.

TomTom

4/3/23, 10:17 AM

1tb is feasible even on a tablet these days.

Score:0

Server

SikorskyS60

4/2/23, 10:06 AM

So basically you want to have your data on a SQL-Server and another SQL-Server which pulls data from the "main" SQL-Server for inserts, updates, selects and so on?

If that's the case, then yes. You can link a database from another server to your MySQL-Server.

0 + 0

Alex Osheter

4/2/23, 10:51 AM

No. There's only one server. The MySQL instance is running on one server, but the data in the database should be someplace else.

SikorskyS60

4/2/23, 2:41 PM

So you want to run miltiple instances of MySQL on one server then? Never tried it myself before but here's a [link](https://ubiq.co/database-blog/how-to-run-multiple-mysql-instances-on-same-machine/) that might help in that case:

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Is it possible to host the database on one server, but the actual data on another?

TH: เป็นไปได้หรือไม่ที่จะโฮสต์ฐานข้อมูลบนเซิร์ฟเวอร์เครื่องหนึ่ง แต่เก็บข้อมูลจริงไว้อีกเซิร์ฟเวอร์หนึ่ง

RO: Este posibil să găzduiți baza de date pe un server, dar datele reale pe altul?

RU: Можно ли разместить базу данных на одном сервере, а сами данные на другом?

VI: Có thể lưu trữ cơ sở dữ liệu trên một máy chủ, nhưng dữ liệu thực tế trên một máy chủ khác không?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.