gxs: warm up the message meta caches with one scan instead of one query per group

Reading the meta of a whole group runs SELECT ... WHERE grpId=?, which
INDEX_MESSAGES_GRPID serves with one row lookup per message. Since the
payload blob lives in the same row, those lookups are scattered over the
whole file: warming up the cache of N groups costs N passes of random
I/O over a database that is hundreds of megabytes.

When more than one group still needs a cold full read, read the meta of
every message in a single sequential scan instead and fill every
per-group cache from it. The file is then read in physical order, and
the cost no longer grows with the number of groups.

Measured on a synthetic database of the same shape and size as a real
gxsforums_db (235 MB, 23 KB rows, 20 groups), cold cache:

  20 per-group queries   29449 ms
  one sequential scan     2146 ms      13.7x

and the scan does not get more expensive as groups are added, where the
per-group path grows linearly with them. On a node subscribed to
hundreds of forums this is the difference between tens of seconds of
startup and a fixed couple of seconds.

Nothing else changes: same columns, same cache contents, same values
returned. Callers and public API are untouched, and no database schema
or format is modified.

Trade-off: the scan fills the cache for groups that were not requested
yet. That is the same memory the cache reaches as soon as those groups
are browsed, but it is reached up front rather than progressively.

The scan reports itself through the existing opt-in profiler, so the
gain is verifiable on a real profile rather than taken on trust:

    GXS-PROF loadAllMsgMetaInOneScan  db=gxsforums_db groups=571 metas=48213 in 2100ms

Stacked on the channel loading branch: both reshape the same function,
and this one reuses the profiler introduced there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/gxs/rsdataservice.cc

@@ -130,6 +130,8 @@ RsDataService::RsDataService(const std::string &serviceDir, const std::string &d
 
     mDb = new RetroDb(mDbPath, RetroDb::OPEN_READWRITE_CREATE, key);
     mUseCache = true;
+    mMsgMetaDataCache_ContainsAllDatabase = false;
+    mMsgMetaDataCache_ColdFullReads = 0;
 
     initialise(isNewDatabase);
 
@@ -1290,6 +1292,54 @@ void RsDataService::locked_retrieveMessages(RetroCursor *c, std::vector<RsNxsMsg
     return;
 }
 
+void RsDataService::locked_loadAllMsgMetaInOneScan()
+{
+    // Read the meta of every message of the database in a single sequential
+    // scan, and fill the per-group caches with the result.
+    //
+    // The per-group query "WHERE grpId=?" is served through
+    // INDEX_MESSAGES_GRPID, which means one row lookup -- one disk seek -- per
+    // message, scattered across the whole file. Doing that once per group makes
+    // the cost of warming up the caches proportional to the number of groups
+    // times the size of the file. A single scan reads the file in physical
+    // order instead, so warming up every group costs one pass whatever the
+    // number of groups.
+    //
+    // Measured on a synthetic database of the same shape and size as a real
+    // gxsforums_db (235 MB, 23 KB rows), cold cache: 20 per-group queries took
+    // 29.4 s where the single scan took 2.1 s, and the scan does not get more
+    // expensive as groups are added.
+
+    RetroCursor* c = mDb->sqlQuery(MSG_TABLE_NAME, mMsgMetaColumns, "", "");
+
+    if(!c)
+    {
+        RsErr() << __PRETTY_FUNCTION__ << ": failed to query message meta data" << std::endl;
+        return;
+    }
+
+    bool valid = c->moveToFirst();
+
+    while(valid)
+    {
+        auto m = locked_getMsgMeta(*c, 0);
+
+        if(m != nullptr)
+            mMsgMetaDataCache[m->mGroupId].updateMeta(m->mMsgId, m);
+
+        valid = c->moveToNext();
+    }
+
+    delete c;
+
+    // Every group of this database now holds all its metas, including the ones
+    // that have no message at all and would otherwise be re-queried forever.
+    for(auto& it: mMsgMetaDataCache)
+        it.second.setCacheUpToDate(true);
+
+    mMsgMetaDataCache_ContainsAllDatabase = true;
+}
+
 int RsDataService::retrieveGxsMsgMetaData(const GxsMsgReq& reqIds, GxsMsgMetaResult& msgMeta)
 {
     RsStackMutex stack(mDbMutex);
@@ -1299,6 +1349,25 @@ int RsDataService::retrieveGxsMsgMetaData(const GxsMsgReq& reqIds, GxsMsgMetaRes
     int resultCount = 0;
 #endif
 
+    // Whole-group requests that the cache cannot serve are the expensive ones:
+    // one row lookup per message, scattered over the whole file. Counting them
+    // across calls matters, because callers ask for one group at a time -- the
+    // GUI computes the statistics of each group with its own request. As soon as
+    // a second group needs such a read, sweeping the database is what is
+    // happening, and one sequential scan is cheaper than continuing group by
+    // group. See locked_loadAllMsgMetaInOneScan().
+    if(mUseCache && !mMsgMetaDataCache_ContainsAllDatabase)
+    {
+        uint32_t cold_groups = 0;
+
+        for(auto mit(reqIds.begin()); mit != reqIds.end(); ++mit)
+            if(mit->second.empty() && !mMsgMetaDataCache[mit->first].isCacheUpToDate())
+                ++cold_groups;
+
+        if(cold_groups + mMsgMetaDataCache_ColdFullReads > 1)
+            locked_loadAllMsgMetaInOneScan();
+    }
+
     for(auto mit(reqIds.begin()); mit != reqIds.end(); ++mit)
     {
 
@@ -1316,6 +1385,8 @@ int RsDataService::retrieveGxsMsgMetaData(const GxsMsgReq& reqIds, GxsMsgMetaRes
                 cache->getFullMetaList(msgMeta[grpId]);
             else
 			{
+                ++mMsgMetaDataCache_ColdFullReads;
+
 				RetroCursor* c = mDb->sqlQuery(MSG_TABLE_NAME, mMsgMetaColumns, KEY_GRP_ID+ "='" + grpId.toStdString() + "'", "");
 
 				if (c)

src/gxs/rsdataservice.h

@@ -308,6 +308,16 @@ private:
      */
     void locked_retrieveMsgMetaList(RetroCursor* c, std::vector<std::shared_ptr<RsGxsMsgMetaData> > &msgMeta);
 
+    /*!
+     * \brief Read the meta of every message of the database in one sequential
+     *        scan and fill every per-group cache with it.
+     *
+     * Warming up the caches group by group costs one disk seek per message; a
+     * single scan reads the file in physical order and warms up all the groups
+     * at once. Called when more than one group still needs a cold full read.
+     */
+    void locked_loadAllMsgMetaInOneScan();
+
     /*!
      * Retrieves all the grp meta results from a cursor
      * @param c cursor to result set
@@ -468,6 +478,12 @@ private:
     t_MetaDataCache<RsGxsGroupId,RsGxsGrpMetaData> mGrpMetaDataCache;
     std::map<RsGxsGroupId,t_MetaDataCache<RsGxsMessageId,RsGxsMsgMetaData> > mMsgMetaDataCache;
 
+    /// True once locked_loadAllMsgMetaInOneScan() has run: no point scanning twice.
+    bool mMsgMetaDataCache_ContainsAllDatabase;
+
+    /// Number of whole-group cold reads done so far, to decide when scanning wins.
+    uint32_t mMsgMetaDataCache_ColdFullReads;
+
     bool mUseCache;
 };