Occasionally when programming in Java you will need a Map too large to hold in memory. MapDB is an open source Java library that allows you to create a disk based Map (and other Java Collections) very easily.

MapDB is a hybrid of an embedded database engine and the Java Collections framework. It provides Maps, Sets, Lists, Queues, etc. that can be stored in off-heap memory or on disk. MapDB is available via Maven central so it can be easily included in your project. Visit the MapDB home page for more information.

Creating a Disk Based Map in Java

Below is an example of creating a disk based HashMap. You can see in the example that DB and DBMaker objects are used to create the HashMap. Each instance of DB represents a single transaction session, so commit() is needed to actually save the data to disk. Note that multiple Maps, Sets, etc. can be written to the same file.

import org.mapdb.DB;
import org.mapdb.DBMaker;
import org.mapdb.Serializer;
import java.util.Map;

public class MapDbTest {

    public static void writeMapToDisk() {
        //use DBMaker to create a DB object stored on disk
        //provide output location of Map
        DB db = DBMaker.fileDB("/tmp/testMapDB.db").make();

        //use the DB object to create the "myMap" Map
        //set the specific serializers for key and value for better performance
        Map<String, String> map = db.hashMap("myMap", Serializer.STRING, Serializer.STRING).createOrOpen();

        //populate map
        for (int i = 0; i < 1000; i++) {
            map.put("key" + i, "value-" + i);
        }

        //persist changes on disk
        db.commit();

        //close to protect from data corruption
        db.close();
    }

}

Reading a Disk Based Map in Java

Reading a Map that has been previously serialized to disk is very similar to creating a Map for writes. We simply have to specify file containing the Map, as well as the name of the Map to deserialize. Also, there is no need to commit() since we are just reading the data.

import org.mapdb.DB;
import org.mapdb.DBMaker;
import org.mapdb.Serializer;
import java.util.Map;

public class MapDbTest {

    public static void readMapFromDisk() {
        //use DBMaker to create a DB object of HashMap stored on disk
        //provide location
        DB db = DBMaker.fileDB("/tmp/testMapDB.db").make();

        //use the DB object to open the "myMap" HashMap
        Map<String, String> map = db.hashMap("myMap", Serializer.STRING, Serializer.STRING).createOrOpen();

        //read from map
        for (int i = 0; i < 1000; i++) {
            System.out.println(map.get("key" + i));
        }

        //close to protect from data corruption
        db.close();
    }

}

MapDB Maven Dependency

Sample MapDB Maven dependency in pom.xml:

<dependency>
  <groupId>org.mapdb</groupId>
  <artifactId>mapdb</artifactId>
  <version>3.0.5</version>
</dependency>