How can that happen, surely an empty page should not be written at all?

Hi @grandinj, I agree this looks really strange... but it happen nevertheless "randomly" but quite often. Once I get this error on my database, then the DB is corrupted and it is impossible to connect to it afterwards.

The pattern is the following: I load an old H2 database (created with a version 1.4.196) and I apply a bunch of migration statements (the connection gets closed). Then, when I want to connect to the database again, it sometimes fails with the given error above. And sometimes it even fails later, this is unclear when. However, with this fix, there is no more crash at all and it works flawlessly.

I would say that this is relative to the NULL/NOT NULL with column with default values. In the previous version (1.4.199), when a default value was specified on a column, the column was described as not nullable, and that behaviour seems to changed in 1.4.200 (which is why I wanted to update).

Up to 1.4.199, I had no problem and nothing changed on our side since then (the impacted H2 database is used for migration tests).

@ctabin, Thank you for your contribution. I wonder, if you can replicate this problem after following recommended migration path of exporting database into a script and run it in a newly created 1.4.200 database?
There is a known problem in versions up to 1.4.197, which leads to creation of internal “keyless” b-tree nodes with a single child, and we have a few safeguards against it. Just search MVMap.java for “single-childed”. Quite likely, it also can produce empty leaf nodes too. While safeguard, you introduce here, would be desired in such case, it will hide a problem, which we might still have in the latest version. That is why I curious to see whether issue is specific to a prior version or we have to dig deeper.

@andreitokar Thanks for your reply. Actually, the dump was created by using version 1.4.196 and was then exported to a script. Hence, we are now loading only a H2 SQL script with the version 1.4.200, applying some migration statements and that's it.

Previously, I added a guard in the Page constructor to prevent arrays of Key of length 0, but that triggered a lot of exceptions and basically h2 was unable to work 😅

I'll see if I can create a test case, even if I think that will be quite hard because of the randomness of the bug 🤔

Ok, so it seems like 1.4.200 issue then.
Are you dealing with spatial data (MVRTreeMap) by any chance?
I any case, I would advice to turn JVM assertions on (-ea), because it may fail much earlier and give us some clues, otherwise it's not much can be done without a test case.

Seems the same pb as we have here orbisgis/geoclimate#188
We are not able to isolate a test case at this time ;(

@andreitokar You're right, we are using spatial data (with GeoDB 0.9). I tried to do a test case on the default value and NULL/NOT NULL change, but this works fine. I'll try to generate a new one with spatial data.

I think i got it. Was able to replicate empty leaf creation with a simple test. Will post the the fix tonight.
@ctabin, It would be nice if you can test that change before it get merged into master. Unfortunately our test coverage for MVRTreeMap is extremely sparse (no pun intended).

@andreitokar Yep, let me know when I can test it 👍

@andreitokar I think I have a testcase that throws a similar error:

java.sql.SQLException: GeneralError
        at org.h2.message.DbException.convert(DbException.java:352)
        at org.h2.message.DbException.toSQLException(DbException.java:319)
        at org.h2.message.TraceObject.logAndConvert(TraceObject.java:366)
        at org.h2.jdbc.JdbcConnection.close(JdbcConnection.java:461)
        at ch.astorm.h2.test.H2Test.createDump(H2Test.java:128)
        at ch.astorm.h2.test.H2Test.testH2MigrationFromCurrentVersion(H2Test.java:33)
        ...
Caused by: java.lang.AssertionError
        at org.h2.mvstore.MVMap.rewrite(MVMap.java:616)
        at org.h2.mvstore.MVMap.rewritePage(MVMap.java:775)
        at org.h2.mvstore.MVMap.rewrite(MVMap.java:734)
        at org.h2.mvstore.MVMap.rewrite(MVMap.java:748)
        at org.h2.mvstore.MVMap.rewrite(MVMap.java:748)
        at org.h2.mvstore.MVMap.rewrite(MVMap.java:710)
        at org.h2.mvstore.MVStore.compactRewrite(MVStore.java:2137)
        at org.h2.mvstore.MVStore.rewriteChunks(MVStore.java:2026)
        at org.h2.mvstore.MVStore.compact(MVStore.java:2007)
        at org.h2.mvstore.MVStore.compactFile(MVStore.java:1968)
        at org.h2.mvstore.MVStore.closeStore(MVStore.java:1170)
        at org.h2.mvstore.MVStore.close(MVStore.java:1133)
        at org.h2.mvstore.db.MVTableEngine$Store.close(MVTableEngine.java:404)
        at org.h2.engine.Database.closeOpenFilesAndUnlock(Database.java:1545)
        at org.h2.engine.Database.closeImpl(Database.java:1454)
        at org.h2.engine.Database.close(Database.java:1373)
        at org.h2.engine.Database.removeSession(Database.java:1307)
        at org.h2.engine.Session.close(Session.java:963)
        at org.h2.jdbc.JdbcConnection.close(JdbcConnection.java:453)

You can just experiment it by running mvn clean package on the project.

@andreitokar Just for information:

            stmt.execute("DROP TABLE IF EXISTS someTable");
            stmt.execute("CREATE TABLE someTable("
                        + "idpk1 varchar(10) NOT NULL,"
                        + "idpk2 varchar(10) NOT NULL,"
                        + "label varchar(255),"
                        + "status1 varchar(1),"
                        + "status2 varchar(1) DEFAULT '2',"
                        + "status3 varchar(1) DEFAULT '3' NOT NULL,"
                        + "status4 int(11) DEFAULT '0',"
                        + "spatialcol geometry NOT NULL,"
                        + "PRIMARY KEY (idpk1,idpk2)"
                    + ")");
            stmt.execute("CREATE SPATIAL INDEX tbl_spatial_idx ON someTable(spatialcol)");

If I comment out the last statement about the SPATIAL INDEX creation, the test works fine :)

@ctabin, Here it is: PR #2231. It fixes your test case.

@andreitokar Thanks for the fix ! This works fine with it.

What should we do about this MR ? I agree that it only circumvent the problem, but still avoid a corruption in that case. Also can I suggest to add the test case on the spatial data ?

I think we should merge it, because the corruption is already in the wild.

Please, send a license statement as described here to our mailing list

@andreitokar Thanks, that's done ! Can we expect a new release soon with those fixes ? 🙏

Thank you!

H2 is not released too often, but you can build H2 by yourself:
https://h2database.com/html/build.html#building

@katzyn Thanks for the merge. Considering this could break existing database, I would suggest to create a release somewhat quickly 😉

Anyway, thanks for your amazing work on this !