Score:2

Using MySQL raw disk on Windows?

ru flag

I'm having trouble getting MySQL 5.7 to use raw disk (a block device, bypassing the filesystem) for its data. My understanding of the documentation is that basically, first time mysqld is executed, the config needs to have the newraw keyword in the value for innodb_data_file_path, and should be replaced with just raw for 2nd execution.

[mysqld]
innodb_data_home_dir    =
innodb_data_file_path   = //./E::512Gnewraw

default-time-zone       = SYSTEM
basedir                 = C:/MySQLSrv/
tmpdir                  = F:/MySQLTemp/
datadir                 = F:/MySQLData-raw/
pid-file                = mysql.pid
port                    = 3306
socket                  = MySQL
enable-named-pipe       = 1
skip-external-locking
key_buffer_size         = 16M
innodb_log_file_size = 26547M
max_allowed_packet      = 90M
table_open_cache        = 64
sort_buffer_size        = 512K
net_buffer_length       = 8K
read_buffer_size        = 256K
read_rnd_buffer_size    = 512K
myisam_sort_buffer_size = 8M
innodb_page_size = 64K

The E: and F: are partitions on a large (~35 TB) vdisk - GPT partitioning, first partition is set to "unformatted" by Windows' disk management, and the 2nd one is a conventional NTFS at 4 TB. (I set innodb data file size to 512 GB just temporarily, after failing to get this working with 4TB, which is what I actually want for a test.) I'm essentially starting the MySQL bringup from scratch, so my first execution of mysqld was with the --initialize-insecure option. This took a while (about as long as I'd expect, given the size configured for the data file and observed disk transfer rate in Task Manager) and eventually produced the usual file structure under the configured datadir. This includes the mysql directory with several *.ibd files, which I'm not sure is supposed to happen. Still, no error in mySQL logs, and the progress line that ends at 524200 suggests that the tablespace was initialised.

2021-10-21T09:47:55.686131Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-10-21T09:47:55.692239Z 0 [Warning] InnoDB: innodb-page-size has been changed from the default value 16384 to 65536.
 100 200 300 400 500 <snip> 523900 524000 524100 524200
 100 200 300 400 500 <snip> 26200 26300 26400 26500
 100 200 300 400 500 <snip> 26200 26300 26400 26500
2021-10-21T09:56:54.318875Z 0 [Warning] InnoDB: New log files created, LSN=130881
2021-10-21T09:56:54.471343Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2021-10-21T09:56:54.533557Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 389cd84c-3255-11ec-8327-005056971dce.
2021-10-21T09:56:54.536035Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2021-10-21T09:56:56.339721Z 0 [Warning] CA certificate ca.pem is self signed.
2021-10-21T09:56:56.436944Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.

So I switch the option to raw, and launch the service (I already had it created from before). The service terminates immediately, logging that it wanted to create a tablespace (again?) because it couldn't find it, and this is a bad idea if redo logs exist - which they did, the --initialize-insecure option created them, as expected.

2021-10-21T10:08:09.625276Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-10-21T10:08:09.625424Z 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled
2021-10-21T10:08:09.625800Z 0 [Note] MySQL (mysqld 5.7.29) starting as process 3356 ...
2021-10-21T10:08:09.634217Z 0 [Warning] InnoDB: innodb-page-size has been changed from the default value 16384 to 65536.
2021-10-21T10:08:09.634627Z 0 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
2021-10-21T10:08:09.634873Z 0 [Note] InnoDB: Uses event mutexes
2021-10-21T10:08:09.635033Z 0 [Note] InnoDB: _mm_lfence() and _mm_sfence() are used for memory barrier
2021-10-21T10:08:09.635262Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-10-21T10:08:09.636051Z 0 [Note] InnoDB: Number of pools: 1
2021-10-21T10:08:09.636460Z 0 [Note] InnoDB: Not using CPU crc32 instructions
2021-10-21T10:08:09.638441Z 0 [Note] InnoDB: Initializing buffer pool, total size = 52G, instances = 16, chunk size = 128M
2021-10-21T10:08:10.597917Z 0 [Note] InnoDB: Completed initialization of buffer pool
2021-10-21T10:08:11.118090Z 0 [Note] InnoDB: The first innodb_system data file '//./E:' did not exist. A new tablespace will be created!
2021-10-21T10:08:11.118733Z 0 [ERROR] InnoDB: redo log file '.\ib_logfile0' exists. Creating system tablespace with existing redo log files is not recommended. Please delete all redo log files before creating new system tablespace.
2021-10-21T10:08:11.119256Z 0 [ERROR] InnoDB: InnoDB Database creation was aborted with error Generic error. You may need to delete the ibdata1 file before trying to start up again.
2021-10-21T10:08:11.336787Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2021-10-21T10:08:11.337058Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2021-10-21T10:08:11.337286Z 0 [ERROR] Failed to initialize builtin plugins.
2021-10-21T10:08:11.337474Z 0 [ERROR] Aborting

2021-10-21T10:08:11.337615Z 0 [Note] Binlog end
2021-10-21T10:08:11.337839Z 0 [Note] Shutting down plugin 'CSV'
2021-10-21T10:08:11.338806Z 0 [Note] MySQL: Shutdown complete

What am I missing here? Why is MySQL not finding the raw disk data file after first initializing it?

Score:0
jp flag

It appears you are not following this specific detail from the documentation listed.

The //./ corresponds to the Windows syntax of \.\ for accessing physical drives.

Windows has different syntax than Linux OS.

ru flag
You're misinterpreting what that detail says - backslashes are the windows-native notation, but MySQL expects forward slashes in the config file as it uses backlash as escape character. Trying to use backslash results in an error: ```2021-10-22T19:29:29.426724Z 0 [ERROR] InnoDB: File \.\E:: 'open' returned OS error 223. Cannot continue operation```
ru flag
And just for the sake of completeness, iI also tried using backslashes in escape mode, ie. entering `\\\\.\\E:` in the config file. The result was pretty much identical to that described in the question.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.