Humble Game Info Userscript: async, await and Promises (Part 2)

Promise you will use Promises

Context

I am writing a userscript for the Humble Bundle library, so that it shows a game description below games for when one needs a little reminder of what what the game is about. In Part 1 we made a proof-of-concept that pulled descriptions from the steam web API. Now we’re going to see if we can do it a little better by introducing methods for storing values we’ve looked up (caching).

Storing Values

We want to have the userscript store values (like the massive array of appids) so that we don’t need to re-fetch them.

the GM_getValue / setValue / listValues / deleteValue methods will be helpful here. (API docs) These manage storage, doing what you would expect from the method names- get/set/list/delete. Before we start storing we’ll set up a couple of helper methods to manage and inspect- listAllCache() and clearAllCache() and bind those to some hotkeys (see below) to aid testing later.

Note: it’s GM_deleteValue, not GM_deleteValues ! That little typo cost me a while, since the eye skips over ‘deleteValues’ as it looks right enough (if it was deleteValuej it might have stood out more); and strangely ViolentMonkey didn’t seem to complain about the typo in the @grant section.

Since we’re storing things, we’re doing a form of caching; so it makes sense to re-order things to hit the cache first, then perform lookups if there is a miss. That is, go from this (as a mental model of flow):

1. Get game title from element on page we clicked
2. Pull in list of steam appids as JSON
3. Look up appid by game title
4. Make request to fetch game info (description) based on appid
5. Display result for the lookup

to something more like:

1. Get game title from element on page we clicked
2. Look up game in localstorage, display results if so
3. If not, look up appid to request from steam api and update local storage
4. If no appid in cache, bring that in too

This brings us closer to where we want to be for a release, but before then there’s a couple of things still to do: some refactoring, and handling errors better.

Gib Me Teh Codes

Once again, the below is presented not for actual use, but to illustrate changes discussed.

// ==UserScript==
// @name Humble Game Info
// @version 0.1.0
// @description Fetch game info for games in Humble Bundle library
// @author bertiebaggio
// @require file:///home/robert/code/humblegameinfo/HumbleGameInfo.user.js
// @require https://ajax.googleapis.com/ajax/libs/jquery/3.7.0/jquery.min.js
// @require  https://gist.github.com/raw/2625891/waitForKeyElements.js
// @require https://cdn.jsdelivr.net/npm/@violentmonkey/shortcut@1
// @match https://www.humblebundle.com/home/library*
// @run-at document-idle
// @grant GM_xmlhttpRequest
// @grant GM_listValues
// @grant GM_getValue
// @grant GM_setValue
// @grant GM_deleteValue
// ==/UserScript==

/*
  Purpose: to add a short game description to Humble Store library page

  Game description is pulled from the unauthenticated Steam web API if game title has a corresponding appID.

  External libs:
   - jQuery
   - waitForKeyElements (for games list loading in)
   - @violentmonkey/shortcut (for debug hotkeys)
  */
var dummyGameInfo = { gameTitle: "Placeholder Game Title",
                      gamePublisher: "Placeholder Game Publisher"
                    };

var dummyGameDetails = { gameDescription: "This is a placeholder description. If you're seeing this you're either testing or something has gone wrong",
                         gameTags: ["FPS", "Open World"],
                       };

var gameDetailsContainer = "<div id='gamedetails' class='gameinfo' style='margin-top: 1em;'><p>Game info goes here!</p></div>";

function clearAllCache ( ) {
    for (const gmkey of GM_listValues()) {
        console.log(`deleting ${gmkey}`);
        GM_deleteValue( gmkey );
    }
}

function listAllCache ( ) {
    for (const gmkey of GM_listValues()) {
        console.log(`${gmkey}: ${GM_getValue( gmkey )}`);
    }
}

async function makeJSONRequest( uri ) {
    return new Promise((resolve, reject) => {
        GM_xmlhttpRequest({
            method: "GET",
            url: uri,
            anonymous: true,
            responseType: 'json',
            onload: response => resolve(response),
            onerror: error => reject(error),
        });
    });
}

/**
 * Add game description to page on RHS
 */
function addGameDescription ( ) {
    // TODO- rewrite this or rename it to be more descriptive
    // Ensure we have somewhere to put the game into by checking for presence of RHS being open
    detailsView = $("div.details-view");
    if (detailsView.length == 1) {
        detailsView.append(gameDetailsContainer);
    }
    let gameInfo = getGameInfoFromPage();

    // set a placeholder description
    updateGameDetailsOnPage(dummyGameDetails.gameDescription);

    // async update on-page
    console.log("Updating page");
    var gameDetails = getGameDetails(gameInfo);
    gameDetails.then(details => {
        let gD = {};
        gD.gameDescription = details;
        updateGameDetailsOnPage(gD);
    });
}

/**
 * Add game description to the visible #gamedetails div
 * @param {Object} gameDetails Object with .gameDescription property
 */
function updateGameDetailsOnPage ( gameDetails ) {
    let gameInfoNode = $("div#gamedetails");
    gameInfoNode.find("p").remove();  // take away placeholder text
    gameInfoNode.find("h3").remove();  // take away placeholder heading
    gameInfoNode.append("<h3><i class='hb hb-info'></i>Description</h3>");
    gameInfoNode.append(`<p>${gameDetails.gameDescription}</p>`);
}

/**
 * Extract game details (ie description and genres) from response object
 * @param {Object} response Object returned from XHR
 * @returns {Object} {gameDescription: String, gameGenres: Array} TODO check latter
 */
function steamGetGameDetailsFromResponse ( response ) {
    console.log(`Got response from steamapi`);
    // appid = response.context.appid;  // needed as json object is returned under appid.data
    // NOTE: technically it's not needed as we could extract the appid from response.finalUrl
    let appid = response.finalUrl.split("=", 2)[1];

    gameDetails = {
        gameDescription: response.response[appid].data.short_description,
        gameGenres: response.response[appid].data.genres,
    };

    return gameDetails;
}

async function steamFetchGameByAppID ( appid ) {
    let baseURL = "https://store.steampowered.com/api/appdetails?appids=";
    let requestURL = baseURL.concat(appid);

    let gamePromise = new Promise( async function(resolve, reject) {
        console.log(`Fetching from ${requestURL}`);
        let response = await makeJSONRequest(requestURL);
        if (response) {
            // response.context = {}; // no longer needed as we extract appid from response.finalUrl
            // response.context.appid = appid;
            let gameDetails = steamGetGameDetailsFromResponse(response);
            resolve(gameDetails);
        } else {
            reject(error);
        }
    });

    let gameDetails = await gamePromise;
    return gameDetails;
}

/**
 * Store list of {appid: name} pairs in localstorage
 * ie _steamappids = { [ {12345: "Example Game"}, {67890: "Another Game"} ] };
 * @param {Object} response response object from XHR
 */
async function steamCacheAppIDsFromResponse ( response ) {
    console.log("Caching appids");
    let appidstocache = new Array();
    for (const appentry of response.response.applist.apps) {
        if (appentry.name.length && appentry.appid) {
            appidstocache.push(appentry);
        }
    }
    console.log(`Storing ${appidstocache.length} appids`);
    await GM_setValue("_steamappids", appidstocache);
}

/**
 * get a Steam AppID for querying Steam public API
 */
async function steamGetAppIDFromTitle ( gameTitle ) {
    function getIDByTitle( gameTitle ) {
        return GM_getValue("_steamappids").filter(
            function(app){ return app.name == gameTitle; }
        );
    }

    appid = getIDByTitle(gameTitle);
    if (appid.length) {
        appidnice = appid[0].appid;
        console.log(`Found app ID: ${appidnice}`);
        return appidnice;
    }

}

async function steamFetchAppIDs () {
    URI = "file:///home/robert/downloads/steamappids2.json";
    let appidPromise = new Promise(async function(resolve, reject) {
        let response = await makeJSONRequest(URI);
        if (response) {
            steamCacheAppIDsFromResponse(response);
            resolve("done");
        } else {
            reject(error);
        }
    });

    let result = await appidPromise;
    return result;
}

/**
 * Wrapper function for fetching game details from steam
 * @return {Promise:String} Description of game from Steam API
 */
async function steamFetchGameDetailsByTitle ( gameTitle ) {
    // Ensure _steamappids are populated for lookup
    let _steamappids = await GM_getValue("_steamappids", -1);
    if (_steamappids == -1 || _steamappids == 'undefined') {
        console.log(`Updating steam appid cache`);
        isdone = await steamFetchAppIDs();
        console.log(`Cach update is: ${isdone}`);
    }
    // pull game info
    console.log("Pulling game info...");
    console.log("Getting ID From Title...");
    let steamappid = await steamGetAppIDFromTitle(gameTitle);
    let gameInfo = await steamFetchGameByAppID(steamappid);
    return gameInfo;
}

async function getGameDetails ( gameInfo ) {
    console.log(`Fetching game info for ${gameInfo.gameTitle}`);
    console.log(`Checking cache...`);
    let hgiGames = await GM_getValue("hgiGames", -1);
    if (!(hgiGames == -1 || hgiGames == 'undefined')) {
        if (gameInfo.gameTitle in hgiGames) {
            return hgiGames[gameInfo.gameTitle];
        } else {
            console.log("Cache miss! Updating...");
            let gameInfo2 = await steamFetchGameDetailsByTitle(gameInfo.gameTitle);
            let description = gameInfo2.gameDescription;
            hgiGames[gameInfo.gameTitle] = description;
            GM_setValue("hgiGames", hgiGames);
        }
    } else {
        hgiGames = {};
        console.log("No game data structure in cache. Fetching game description and updating");
        let gameInfo2 = await steamFetchGameDetailsByTitle(gameInfo.gameTitle);
        let description = gameInfo2.gameDescription;
        hgiGames[gameInfo.gameTitle] = description;
        GM_setValue("hgiGames", hgiGames);
    }
    console.log(`hgiGames[${gameInfo.gameTitle}]: ${hgiGames[gameInfo.gameTitle]}`);
    return hgiGames[gameInfo.gameTitle];
}


/**
 * Get game title and publisher from page
 * Note that we don't actually use publisher at present with steam API
 * @return {Object} { gameTitle: title, gamePublisher: publisher }
 */
function getGameInfoFromPage ( ) {
    gameTitleNode = $("div.details-heading").find(".text-holder").find("h2");
    if (gameTitleNode.length) {
        gameTitle = gameTitleNode.text();
    }
    gamePublisherNode = $("div.details-heading").find(".text-holder").find("p");
    if (gamePublisherNode.length) {
        gamePublisher = gamePublisherNode.text();
    }
    return { gameTitle: gameTitle, gamePublisher: gamePublisher};
}

waitForKeyElements("div.subproduct-selector", function( jNode ){
    // Main entry point - adds listener to game entry divs on LHS
    jNode.on("click", { node: jNode }, addGameDescription);
});

// utility stuff
$(window).on("load", () => {
    console.log("HGI: Setting example cache value");
    GM_setValue("foo", "bar");
    console.log("HGI: Setting up key bindings");
    VM.shortcut.register('c-c', () => {
        console.log("Clearing cache...");
        clearAllCache();
    });
    VM.shortcut.register('c-l', () => {
        console.log("Listing cache...");
        listAllCache();
    });
});

Questions / Issues Arising During Development

How do I bind a hotkey?

For debugging purposes, hotkeys are useful action triggers, such as for clearing cache and debug printing to console. ViolentMonkey has a library for this: @violentmonkey/shortcut

// @require https://cdn.jsdelivr.net/npm/@violentmonkey/shortcut@1
// ...
VM.shortcut.register('c-c', () => {
    console.log("Clearing cache...");
    clearAllCache();
});

How do I wait for something to finish before continuing?

This implies asynchronous functions. Use await to wait for an asynchronous function to complete before continuing. Note that the containing function itself has to be declared async. For example:

console.log(`Checking cache...`);
if (!GM_getValue("_steamappids").length) {
    console.log(`Updating steam appid cache`);
    await steamFetchAppIDs(gameInfo); // TODO: remove gameInfo / context
}
// pull game info
console.log("Pulling game info...");
steamGetAppIDFromTitle(gameInfo.gameTitle);

But wait! That doesn’t actually work. This is where I need to wrap my head around javascript’s Promises. There’s mdn docs on them which are helpful. More promises (w3schools) and more promises (freecodecamp). There’s also the quite good ‘modern javascript tutorial’ on promise basics and async/await (plus the async overview). Last-but-not-least, there’s this converted SO answer on implementing Promises, which builds them from the ground up.

Getting into a Promise & await/async mindset takes some time and some reading! A key takeaway is understanding that simply declaring a function async and calling it with awaitdoesn’t magically make the calling code wait for that function to finish. All that declaring a function async does is make it return whatever it was going to return but wrapped nicely in a Promise.

My simplified high-level overview is that Promises are fancy syntatic-sugar-wrapped callbacks. Analogies commonly use ‘producers’ and ‘consumers’, so Promises let you say:

• Consumer: “I need <this thing>. Please supply it!”
• Producer: “I’m going to go do make (optionally asynchronously) <this thing>. You’ll either get it or not, I’ll let you know either way.”
• Consumer: “Thanks, once I get <this thing> I’ll either use it or figure out what I’ll do if I don’t.”

Once again I am indebted to Bob for his helpful discussion of Promises.

Why doesn’t ViolentMonkey appear in my toolbar? (Solved!)

Clicking the puzzle piece icon in the toolbar let me pin ViolentMonkey to the toolbar!

Next Steps

Now that the userscript does things in the correct order (ie hitting cache first), we can look at making handling errors more robust (like not trying to look up a game that isn’t in the list of appids) while making the code more consistent.