Memory

We recently had the situation that a customer complained about the Oom killer terminating the MariaDB database instance from time to time. The MariaDB database configuration was sized quit OK (about 50% of RAM was used for the database) but they did not have swap configured.

When we checked the memory for the specific mysqld process we found that VSZ was about 80 Gibyte (on a 64 Gibyte machine) and the RSS size was about 42 Gibyte. The very high VSZ value in combination with a lacking swap space and Oom …

See also our former articles:

• Do not underestimate performance impacts of swapping on NUMA database systems
• MariaDB and MySQL swap analysis

When we do analysis of MariaDB Database servers we also check the memory (RAM and Swap) available:

# free --kilo --wide
              total        used        free      shared     buffers       cache   available
Mem:       16106252     4329952      703356      199008      307872    10765072    11042748
Swap:      31250428      528684    30721744

The values for …

# free
              total        used        free      shared  buff/cache   available
Mem:       16106332     3914228      367072      296508    11825032    11441608
Swap:      31250428      182924    31067504

# cat /proc/meminfo | grep ^Swap
SwapCached:        10056 kB
SwapTotal:      31250428 kB
SwapFree:       31067504 kB

# for file in /proc/*/status ; do
  awk '/VmSwap|Name/{
    if ( $2 != "0" )
      printf $2 " " $3
  }END{ print ""}' $file
done | grep kB | sort -k 2 …

Introduction

In his presentation about PBXT at the DOAG Conference 2010 Paul McCullagh was speaking about memory resident PBXT tables. They will be available in version 1.1 of the PBXT Storage Engine Plugin. Memory resident PBXT tables should have similar characteristics like normal MySQL MEMORY</span> tables</a>. But in addition to the MEMORY tables they are transactional and can handle BLOB</span> and TEXT attributes.

An alternative to this feature would be the MySQL dynamic heap row …

This is an automated analysis of the DBI:mysql:database=foodmart;host=localhost database for migration into MySQL Cluster. No warranty is made to the accuracy of the information.

This information should be valid for MySQL 4.1 and 5.0. Since 5.1 is not a final release yet, the numbers should be used as a guide only.

5.1-dd is for tables stored on disk. The ndb_size.pl estimates are experimental and should not be trusted. Notably we don’t take into account indexed columns being in DataMemory versus …

MySQL Cluster is pretty fast. The reason for this is, that it is completely memory based. Nowadays memory is still, in contrary to disk, limited to your systems. Thus, before installing a MySQL Cluster you have to calculate the amount of memory you need.

To say it in advance: You should consider to only use 64-bit Linux system with huge amount (4 - 64 GB) of RAM!

In release 5.1 MySQL Cluster became disk based. Now you have the possibility to swap out some data to disk. How much it is we will probably see a …

As you probably have experienced yet MySQL does not always provide all internal information as you might want to have them and as you are used to have from other RDBMS.

MySQL plans to improve this implementing the/a performance schema and its probably already partly done in MySQL 5.4. But who knows when this will be finished and what it contains at all…

What is not provided to me I want to gather myself… But how? Other RDBMS provide interfaces to attach applications directly to their memory to …

The following tools we use sometimes for our consulting engagements…

Caution: These tools are NOT for production use! Use with care!

Tools

• allocate.c
• backslashG2table.pl
• general2bench.pl
• memuse.pl
• memwaster.c
• MySQLprofiler
• pointer_size.c
• alter_engine.pl
• commit_demo.pl
• test.sh
• vm_mon.sh
• read_process_memory.c
• read_process_memory.sh
• cluster_initial_test.pl
• mem_map.pl
• csv_converter.pl
• mem_tracker.sh
• filesystem_table.php

Sample Databases

• MySQL world: world.tgz (92 kbyte, runs with version 4.1 (tested …

As you probably have experienced yet MySQL does not always provide all internal information as you might want to have them and as you are used to have from other RDBMS.

MySQL plans to improve this implementing the/a performance schema and its probably already partly done in MySQL 5.4. But who knows when this will be finished and what it contains at all…

What is not provided to me I want to gather myself… But how? Other RDBMS provide interfaces to attach applications directly to their memory to …

A customer has recently experienced a possible memory leak in its NDB-API application. What he did was something like:

# ps aux | grep <pid>

over time and then he saw the RSS increasing. When he would have had a look a little longer he would have seen that the RSS consumption would increase up to a certain level and then becomes stable. Which is the expected behavior.

But how to explain to the customer that his application, which was in fact not doing anything, consumes more RSS?

With a diff over …