18. Install
ColumnStore初期設定
# /usr/local/mariadb/columnstore/bin/postConfigure
Select the type of System Server install [1=single, 2=multi] (2) > 1
Enter System Name (columnstore-1) > columnstore-1
Select the type of Data Storage [1=internal, 2=external] (1) > 1
Enter the list (Nx,Ny,Nz) or range (Nx-Nz) of DBRoot IDs assigned to module 'pm1' (1) > 1
===== Performing Configuration Setup and MariaDB ColumnStore Startup =====
...
Enter 'mcsmysql' to access the MariaDB ColumnStore SQL console
Enter 'mcsadmin' to access the MariaDB ColumnStore Admin console
27. インポート
cpimport 実行例
[myuser@srv1~]# cpimport sample table1 /tmp/table1.csv –s ‘,’ –E ‘”’
Locale is : C
Column delimiter : ,
Enclosed by Character : ”
Using table OID 3278 as the default JOB ID
Input file(s) will be read from : /root/tmp
…
2017-08-31 16:16:37 (18403) INFO : For table sample.table1: 10000 rows
processed and 10000 rows inserted.
…
2017-08-31 16:16:37 (18403) INFO : Bulk load completed, total run time :
1.22974 seconds
DB TABLE File path option
30. インポート
INSERT SELECT, LOAD DATA INFILE
共通の注意点
• ロードするデータファイルサイズが大きい場合
バッファサイズ不足でロードに失敗する
• ERROR 1815 (HY000) at line 1 in file: ’table1.sql': Internal error: CAL0006: IDB-
2008: The version buffer overflowed. Increase VersionBufferFileSize or limit
the rows to be processed.
33. インポート
VersionBufferFileSize確認/変更
[myuser@srv1~]# cd /usr/local/mariadb/columnstore/bin/
[myuser@srv1~]# configxml.sh getconfig VersionBuffer VersionBufferFileSize
Current value of VersionBuffer / VersionBufferFileSize is 1GB
[myuser@srv1~]# configxml.sh setconfig VersionBuffer VersionBufferFileSize 10GB
Old value of VersionBuffer / VersionBufferFileSize is 1GB
VersionBuffer / VersionBufferFileSize now set to 10GB
section variablecommand
50. TPC-H Benchmark
InnoDBデータベース作成
MySQL Host 127.0.0.1
MySQL Port 3306
MySQL User root
MySQL User password rootのパスワード
MySQL Database tpch
Data Warehouse Storage engine innodb
Scale Factor 10
Virtual Users Build Schema 1
64. TPC-H Benchmark
クエリ書き換え
Query # Error
Query 2 ERROR 1815 (HY000): Internal error: IDB-3012: Scalar filter and semi join are not from the same
pair of tables.
Query 5 ERROR 1815 (HY000): Internal error: IDB-1003: Circular joins are not supported.
Query 17 ERROR 1815 (HY000): Internal error: IDB-3012: Scalar filter and semi join are not from the same
pair of tables.
Query 19 ERROR 1815 (HY000): Internal error: IDB-1000: 'lineitem' and 'part' are not joined.
ColumnStoreで実行する場合クエリを書き換える必要がある
73. TPC-H Benchmark
my.cnf 編集
# cd /usr/local/mariadb/columnstore/mysql
# vi my.conf
…
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 20GB
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 100M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50
InnoDBでベンチマーク時のみ有効にする
75. TPC-H Benchmark
ベンチマーク実行
MySQL Host 127.0.0.1
MySQL Port 3306
MySQL User root
MySQL User password rootのパスワード
MySQL Database tpch or tpch_mcs
Data Warehouse Storage engine innodb
Scale Factor 10
Virtual Users Build Schema 1