Score:0

Recomendations for Data Transfer between heterogeneous sources/destinations

cn flag

I apologize if this is not the right forum for this kind of question.

We need to transfer data between DB Servers (MySQL, MSSQL), CSV files, API Sources (Web Services), etc. and we are looking into a program/source that can do that for us. (Currently we have custom made programs, some SSIS flows, etc.)

Any recommendations?

Thank you.

Score:0
ua flag

There are many products (or tools within a given engine) for transferring data. Usually, it involves "exporting" the data from one engine and "importing" into the other.

For MySQL as a destination, MySQL Workbench handles several sources.

Many engines have a way to read a CSV file, or even treat it as a table.

You should ask about a specific pair of source and destination.

Daniel avatar
cn flag
Thank you, but since we have many sources/destinations, we are looking into a tool/software where we can consolidate all those imports/exports into one place instead of going into multiple apps (like SSIS, SQL Agent, Custom Programs, Azure Logic Apps/Functions, etc). Like a list of "tasks", each task has a "source", "transform", "destination" and "schedule/trigger".
ua flag
@Daniel - For further discussion, please provide short samples of the various source formats. All CSV-like inputs (even with tabs / commas / etc; diff number of header lines, etc) can be handled by essentially one mechanism. What _other_ formats do you have? For example, XML is the pits, but XSLT may be the answer.
Daniel avatar
cn flag
We transfer data from CSV / JSON located either in the filesystem or in a FTP/SFTP to MySQL, MSSQL and Web Services (REST), Web Services to MSSQL (this might be difficult because we use custom libraries, but it would be nice), a some more... the idea came up because we use a "online reporting service", and they have a very nice desktop program that allow us to get data from multiple sources (a lot), transform it and send it to it, unfortunately its destination is fixed to that product, we would like something generic. Thank you.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.