Unable to create tables with ROW_FORMAT=COMPRESSED

Question

I'm attempting to install Nextcloud on MariaDB 10.4.11-MariaDB-1:10.4.11+maria~bionic in a docker container (mariadb:latest sha256:2f11cf2ec18988aec8346a5cf528d69ac3f0f4fc02af79ba28f4fd47b7778d6f).

First thing the installer does is attempt to create a table with ROW_FORMAT=COMPRESSED in the DDL, which produces an error 140 "Wrong create options".

If I omit the row_format parameter, the table gets created, but with the DYNAMIC row_format.

What's going on here? Did ROW_FORMAT=COMPRESSED get deleted at some point, or is the version of MariaDB docker is pulling not compiled with it in?

Short of modifying Nextcloud's SQL DDLs, what can I do here?

Edit:

Finally in a position to add more info about my system, here goes:

First, my my.cnf

[mysqld]
innodb_large_prefix=on
innodb_file_per_table=on
innodb_doublewrite=off
innodb_page_size=64k
innodb_file_format=barracuda

innodb_compression_algorithm=lz4
innodb_compression_default=ON

This results in a startup that contains lines:

2020-01-11 07:10:48+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.4.11+maria~bionic started.
2020-01-11 07:10:57+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2020-01-11 07:10:58+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.4.11+maria~bionic started.
2020-01-11  7:10:58 0 [Note] mysqld (mysqld 10.4.11-MariaDB-1:10.4.11+maria~bionic) starting as process 1 ...
2020-01-11  7:10:58 0 [Warning] The parameter innodb_file_format is deprecated and has no effect. It may be removed in future releases. See https://mariadb.com/kb/en/library/xtradbinnodb-file-format/
2020-01-11  7:10:58 0 [Note] InnoDB: innodb_page_size=65536
2020-01-11  7:10:58 0 [Note] InnoDB: Using Linux native AIO
2020-01-11  7:10:58 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-11  7:10:58 0 [Note] InnoDB: Uses event mutexes
2020-01-11  7:10:58 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-11  7:10:58 0 [Note] InnoDB: Number of pools: 1
2020-01-11  7:10:58 0 [Note] InnoDB: Using SSE2 crc32 instructions
2020-01-11  7:10:58 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
2020-01-11  7:10:58 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
2020-01-11  7:10:58 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-11  7:10:58 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-11  7:11:06 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-11  7:11:06 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-11  7:11:06 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-11  7:11:06 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-11  7:11:06 0 [Note] InnoDB: Waiting for purge to start
2020-01-11  7:11:06 0 [Note] InnoDB: 10.4.11 started; log sequence number 202590810; transaction id 682452
2020-01-11  7:11:06 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2020-01-11  7:11:06 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-11  7:11:07 0 [Note] Server socket created on IP: '::'.
2020-01-11  7:11:08 0 [Note] Reading of all Master_info entries succeeded
2020-01-11  7:11:08 0 [Note] Added new Master_info '' to hash table
2020-01-11  7:11:08 0 [Note] mysqld: ready for connections.
Version: '10.4.11-MariaDB-1:10.4.11+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
2020-01-11  7:11:30 0 [Note] InnoDB: Buffer pool(s) load completed at 200111  7:11:30

Since I had to alter the create table DDL to even run, this is an example of one such table that actually got created:

CREATE TABLE `oc_users` (
  `uid` varchar(64) COLLATE utf8mb4_bin NOT NULL DEFAULT '',
  `displayname` varchar(64) COLLATE utf8mb4_bin DEFAULT NULL,
  `password` varchar(255) COLLATE utf8mb4_bin NOT NULL DEFAULT '',
  `uid_lower` varchar(64) COLLATE utf8mb4_bin DEFAULT '',
  PRIMARY KEY (`uid`),
  KEY `user_uid_lower` (`uid_lower`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin `PAGE_COMPRESSED`='ON' 

Note that only PAGE_COMPRESSED was a valid option, the ROW_FORMAT=COMPRESSED option just fails regardless.

The SHOW GLOBAL STATUS command: https://pastebin.com/LzjsAvAX

And the SHOW VARIABLES command: https://pastebin.com/64zpmh2Z

The system I'm running on has 8 GB of RAM, of which MariaDB is using approx 170 MB.

Answer 1

Since you posted the VARIABLES and GLOBAL STATUS, here is an analysis. (Not much to say.)

Observations:

• Version: 10.4.11-MariaDB-1:10.4.11+maria~bionic
• 8 GB of RAM
• Uptime = 1d 04:38:55
• Are you sure this was a SHOW GLOBAL STATUS ?
• You are not running on Windows.
• Running 64-bit version
• You appear to be running entirely (or mostly) InnoDB.

The More Important Issues:

innodb_buffer_pool_size is considerably bigger than the amount of data you have. Leave it alone unless you are having memory issues, in which case, lower it.

myisam_sort_buffer_size is rather high, especially since you appear not to be using MyISAM.

0.6 queries/second -- very low. I assume you are not having performance problems?

Details and other observations:

( (key_buffer_size / 0.20 + innodb_buffer_pool_size / 0.70) ) = ((128M / 0.20 + 256M / 0.70)) / 8192M = 12.3% -- Most of available ram should be made available for caching. -- http://mysql.rjweb.org/doc.php/memory

( open_files_limit ) = 935 -- ulimit -n -- To allow more files, change ulimit or /etc/security/limits.conf or in sysctl.conf (kern.maxfiles & kern.maxfilesperproc) or something else (OS dependent)

( innodb_lru_scan_depth ) = 1,024 -- "InnoDB: page_cleaner: 1000ms intended loop took ..." may be fixed by lowering lru_scan_depth

( Innodb_buffer_pool_pages_free / Innodb_buffer_pool_pages_total ) = 2,671 / 4096 = 65.2% -- Pct of buffer_pool currently not in use -- innodb_buffer_pool_size (now 268435456) is bigger than necessary?

( Innodb_pages_written / Innodb_buffer_pool_write_requests ) = 92,258 / 218188 = 42.3% -- Write requests that had to hit disk -- Check innodb_buffer_pool_size (now 268435456)

( innodb_page_size ) = 65,536 -- (As of 5.6.4) Applies to all tablespaces. Smaller may help in random access with SSDs. -- (Values other than 16KB are rare; care is needed when using such.)

( Innodb_buffer_pool_bytes_data / innodb_buffer_pool_size ) = 91,619,328 / 256M = 34.1% -- Percent of buffer pool taken up by data -- A small percent may indicate that the buffer_pool is unnecessarily big.

( innodb_doublewrite ) = innodb_doublewrite = OFF -- Extra I/O, but extra safety in crash. -- OFF is OK for FusionIO, Galera, Slaves, ZFS.

( Innodb_os_log_written / (Uptime / 3600) / innodb_log_files_in_group / innodb_log_file_size ) = 34,577,920 / (103135 / 3600) / 2 / 48M = 0.012 -- Ratio -- (see minutes)

( Uptime / 60 * innodb_log_file_size / Innodb_os_log_written ) = 103,135 / 60 * 48M / 34577920 = 2,502 -- Minutes between InnoDB log rotations Beginning with 5.6.8, this can be changed dynamically; be sure to also change my.cnf. -- (The recommendation of 60 minutes between rotations is somewhat arbitrary.) Adjust innodb_log_file_size (now 50331648). (Cannot change in AWS.)

( default_tmp_storage_engine ) = default_tmp_storage_engine =

( innodb_flush_neighbors ) = 1 -- A minor optimization when writing blocks to disk. -- Use 0 for SSD drives; 1 for HDD.

( sync_binlog ) = 0 -- Use 1 for added security, at some cost of I/O =1 may lead to lots of "query end"; =0 may lead to "binlog at impossible position" and lose transactions in a crash, but is faster.

( innodb_print_all_deadlocks ) = innodb_print_all_deadlocks = OFF -- Whether to log all Deadlocks. -- If you are plagued with Deadlocks, turn this on. Caution: If you have lots of deadlocks, this may write a lot to disk.

( myisam_sort_buffer_size ) = 512M / 8192M = 6.2% -- Used for ALTER, CREATE INDEX, OPTIMIZE, LOAD DATA; set when you need it. Also for MyISAM's REPAIR TABLE. -- Decrease myisam_sort_buffer_size (now 536870912) to keep from blowing out RAM.

( character_set_server ) = character_set_server = latin1 -- Charset problems may be helped by setting character_set_server (now latin1) to utf8mb4. That is the future default.

( local_infile ) = local_infile = ON` -- local_infile (now ON) = ON is a potential security issue

( (query_cache_size - Qcache_free_memory) / Qcache_queries_in_cache / query_alloc_block_size ) = (64M - 64618032) / 1432 / 16384 = 0.106 -- query_alloc_block_size vs formula -- Adjust query_alloc_block_size (now 16384)

( Select_scan / Com_select ) = 10,886 / 22440 = 48.5% -- % of selects doing full table scan. (May be fooled by Stored Routines.) -- Add indexes / optimize queries

( binlog_format ) = binlog_format = MIXED -- STATEMENT/ROW/MIXED. -- ROW is preferred by 5.7 (10.3)

( slow_query_log ) = slow_query_log = OFF -- Whether to log slow queries. (5.1.12)

( long_query_time ) = 10 -- Cutoff (Seconds) for defining a "slow" query. -- Suggest 2

( back_log ) = 70 -- (Autosized as of 5.6.6; based on max_connections) -- Raising to min(150, max_connections (now 100)) may help when doing lots of connections.

( thread_cache_size / Max_used_connections ) = 100 / 1 = 10000.0% -- There is no advantage in having the thread cache bigger than your likely number of connections. Wasting space is the disadvantage.

Abnormally small:

1.2 * Key_blocks_used * 1024 = 0
Acl_database_grants = 1
Acl_users = 4
Aria_pagecache_read_requests = 0.38 /HR
Aria_pagecache_reads = 0.38 /HR
Aria_pagecache_write_requests = 0
Com_insert = 0.52 /HR
Com_show_fields = 0
Com_show_tables = 0
Empty_queries = 93 /HR
Handler_icp_attempts = 1.2 /HR
Handler_icp_match = 1.2 /HR
Handler_read_key = 0.35 /sec
Handler_read_next = 6.9 /HR
Handler_read_next / Handler_read_key = 0.00554
Handler_tmp_update = 0
Handler_tmp_write = 19 /HR
Handler_write = 0.56 /HR
Innodb_buffer_pool_pages_total = 4,096
Innodb_buffer_pool_read_ahead = 0
Innodb_buffer_pool_read_requests = 8.8 /sec
Innodb_data_reads = 49 /HR
Innodb_dblwr_pages_written = 0
Innodb_pages_read = 44 /HR
Innodb_rows_deleted + Innodb_rows_inserted = 0.73 /HR
Innodb_rows_inserted = 0.49 /HR
Innodb_secondary_index_triggered_cluster_reads = 15 /HR
Key_blocks_used = 0
Key_read_requests = 0
Key_reads+Key_writes + Innodb_pages_read+Innodb_pages_written+Innodb_dblwr_writes+Innodb_buffer_pool_pages_flushed = 1.8 /sec
Open_table_definitions = 60
Open_tables = 55
Rows_read = 5.5 /sec
Rows_tmp_read = 19 /HR
Table_locks_immediate = 4 /HR
Table_open_cache_hits = 0.47 /sec
innodb_spin_wait_delay = 4
join_buffer_size * Max_used_connections / _ram = 0.00%
lock_wait_timeout = 86,400
query_cache_limit = 131,072

Abnormally large:

Innodb_buffer_pool_pages_flushed / max(Questions, Queries) = 1.46
Innodb_log_writes / Innodb_log_write_requests = 197.8%
Innodb_num_pages_page_compressed = 2,900
Innodb_num_pages_page_decompressed = 841
Innodb_page_compression_saved = 1.71e+8
group_concat_max_len / _ram = 0.01%
histogram_size = 254
myisam_sort_buffer_size / _ram = 6.2%
optimizer_trace_max_mem_size = 1.05e+6
optimizer_use_condition_selectivity = 4
performance_schema_max_stage_classes = 160
performance_schema_max_statement_classes = 202
query_prealloc_size / _ram = 0.00%

Abnormal strings:

binlog_annotate_row_events = ON
disconnect_on_expired_password = OFF
histogram_type = DOUBLE_PREC_HB
innodb_compression_algorithm = lz4
innodb_fast_shutdown = 1
innodb_use_atomic_writes = ON
log_slow_admin_statements = ON
myisam_stats_method = NULLS_UNEQUAL
old_alter_table = DEFAULT
opt_s__engine_condition_pushdown = off
opt_s__optimize_join_buffer_size = on
optimizer_trace = enabled=off
plugin_maturity = gamma
replicate_annotate_row_events = ON
use_stat_tables = PREFERABLY_FOR_QUERIES
wsrep_debug = NONE
wsrep_load_data_splitting = OFF

Answer 2

Thanks to a back-handed comment, I drilled down into the docs for innodb_page_size, which briefly mentions:

ROW_FORMAT=COMPRESSED is not supported when innodb_page_size is set to 32KB or 64KB.

Since I've tuned the InnoDB page size to match the storage media (i.e. 64 Kb) this meant that ROW_FORMAT=COMPRESSED was automatically (and permanently) disabled, and there is no explanation in-system to explain that.

So, long story short, you must crawl every micrometer of the docs if you intend to do anything, and any optimizations you may attempt will immediately bring you into "Not supported go away" territory. Otherwise, vague and meaningless errors will occur.

Oh well.