mbox data storage management

mbox holds different type of database depending which operating modes:

There are three ways to store and backup mbox data:

By default, all data is stored in the same disk that shares with mboxOS, which has limited storage space. For mlog and mfusion, where there's large amount of data to be stored, it's usually not possible to store on mbox OS disk. So we need to slot in additional HDD (USB drive or built-in SSD/SATA/SAS etc), so that data is stored in the large disk, without impacting OS operational functions.


BACKUP TO LOCAL PC

Login to mbox portal, eg. http://<mboxip>, login as mboxadmin, then password. Once login, go to ADMIN -- > Settings -- Config Backups.

NOTE: by default the backup only saves config related data, such as user accounts, profiles, or syslog alarm rules and mfusion configuration and mbox user/hosts/entities data etc. You can optionally tick to save user access records, or mfusion monitoring data etc.

Once the backup files are created, you can download offline to store on your PC. In case of recovery, you can upload it back and restore your backups.

Please refer to video demo for more comprehensive mbox disaster recovery options.

STORE ON USB OR ADDITIONAL HDD STORAGE

In this example, we have an 1TB USB drive added to mbox (either built-in HDD or USB), so that we can store large amount of Syslog/radius/mfusion data using external disk. In the case of mlog, it already comes with a built-in additional SSD storage HDD, 1TB or 2TB depending on the models. But we should go through the same steps to mount the additional HDD.

PART 1: discover and mount USB storage (or new disk)

there are 3 steps in this part:

1. When a USB/HDD drive is slotted into mbox USB port, it will be automatically discovered and mounted. Check what’s the drive name.

mfusion#mfusion# disk show Disk /dev/sda: 32.0 GB, 32000442368 bytes255 heads, 63 sectors/track, 3890 cylinders, total 62500864 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000213e1  Device Boot      Start         End      Blocks   Id  System/dev/sda1   *        2048    59895807    29946880   83  Linux/dev/sda2        59897854    62531583     1316865    5  Extended/dev/sda5        59897856    62531583     1316864   82  Linux swap / SolarisDisk /dev/sdb: 1024.2 GB, 1024209543168 bytes   <--this is the USB storage........

In above output,

Sometimes, we may have another disk within mbox for special purposes, so the new USB/HDD storage will be detected as /dev/sdc, similarly if you have more disks, they will be detected as /dev/sde, /dev/sdf, etc et.

2. Once you identify the drive name for your USB/HDD storage, create and format the disk. Enter below command and press enter to finish at the end. 

mfusion#mfusion# disk create /dev/sdball data on disk /dev/sdb will be lost. do you want to continue "y" or "n": yCalling ioctl() to re-read partition table.Syncing disks.mke2fs 1.42.5 (29-Jul-2012)Filesystem label=OS type: Linux..........Allocating group tables: done                            Writing inode tables: done                            Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information:          doneplease map new partition /dev/sdb1 to a folder.mfusion#

And issue “disk show” again, you should see the new partition added.

mfusion#mfusion# disk showDisk /dev/sdb: 1000.2 GB, 1000170586112 bytes11 heads, 16 sectors/track, 11099194 cylinders, total 1953458176 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x1d084add  Device Boot      Start         End      Blocks   Id  System/dev/sdb1            2048  1953458175   976728064   83  Linux   <--new partition addedmfusion#

3. Now the new disk will appear as a local partition, then we mount it to a local folder so we can use it to store data. (use the partition name from above "disk show", sometimes could be /dev/sdb1 or /dev/dbc1)

mfusion#mfusion# disk mount /data /dev/sdb1creating a new local folder /data/data is mounted to /dev/sdc1 successfully.

To verify the disk is mounted to local folder and ready for storing data, use “show system disk”

mfusion#mfusion# show system disk Filesystem                                              Size  Used Avail Use% Mounted onrootfs                                                   28G  4.4G   23G  17% /udev                                                     10M     0   10M   0% /devtmpfs                                                   1.6G  224K  1.6G   1% /runtmpfs                                                   5.0M     0  5.0M   0% /run/locktmpfs                                                   3.2G  4.0K  3.2G   1% /run/shmtmpfs                                                   256M   35M  222M  14% /var/log/dev/sdb1                                               917G  285M  870G   1% /datamfusion# 

PART 2: store SQL data to mounted USB/HDD storage

Now the USB/HDD storage is mounted as a local disk and mount to a local folder, ready to use. We configure mysql server to use the mounted folder to store data.

!mfusion mysql-server data-path /data start!

ARCHIVE TO EXTERNAL NAS STORAGE (FTP)

We can optionally backup data to external NAS storage for data archival purposes (configure NAS as a FTP server). This is particularly useful for mlog because it usually expects large amount of log data. Even the additional HDD may not be sufficient to store all data, especially for long-term log retention requirements.


!security log-server log-input 10 accept all data-lifetime 5 ftp ftpuser ftppass <ftpserverip> <folder on ftp, eg. /backup> start!

In above configuration:


mbox# show system backup ftptotal 12K4.0K drwxr-xr-x  2 root  root  4.0K Apr  2 13:33 .4.0K drwxr-xr-x 28 root  root  4.0K Apr  2 21:27 ..4.0K -rw-rw-rw-  1 mysql mysql   53 Apr  2 13:33 Syslog-SystemEvents-20150401.csv.gz