Categories
automation python

Including Contemporaneous Info in my YouTube Workflow

From the Department of Wordy Titles

I have a set of tools that I have written to make interacting YouTube simpler, more straightforward, simplifying my workflow.

In the state it’s in it roughly looks like:

  1. record a bunch of videos
  2. upload the files and leave them in place
  3. run genjson on them to create a JSON template, including a reasonably-spaced publish schedule
  4. run get_ids to associate the JSON entries with the video’s YT videoId
  5. go through the videos, rewatch to decide on title, description and thumbnail frame and include this in the JSON entry
  6. run uploadytfootage to update the metadata

Most of the above is highly automated- even step 2 could be done away with if the default YouTube API quota didn’t limit one to roughly six videos per day.

The most labour-intensive part of the process is step 5. Because of the batch nature of the job, sometimes quite a few videos can pile up. For example, at time of writing I have 45 Hunt: Showdown videos from the past ten days to do.

Getting a short, catchy yet descriptive title and description for each of those will involve reacquainting myself with what those round[s] entailed. So I decided recently that I would try to do some of that work as I go: between rounds of Hunt, write out a putative title and description associated with a video file to another JSON file.

I also capture a short snippet or potential title on a notepad on my desk:

Between those hopefully the process will be a bit easier.

I also cooked up a short script to merge together the two JSON files. The crux of it is the filter that selects from the ‘contemporaneous note’ if it has an associated entry for a file in the generated JSON template list.

We are working with a list of dicts, so a list comprehension is handy. We want to select from the list of dicts an entire dict that matches the filename of the video. Roughly speaking:

next(item for item in json_c if item["file"] = filename)

Docs: list comprehension, next()
SO example: Python list of dictionaries search

If I am able to keep on top of titles and descriptions as I go, the only thing needed will be to find a good thumbnail frame! (though that’s kinda time consuming in itself, perhaps ML could be applied to that…)

Edit: Yes! Deep neural net thumbnails and convolutional neural nets (PDF)

Categories
automation python timesavers video

Rescheduling YouTube Videos using Python

More ‘exactly what it says on the tin’

A couple weeks ago, I had to renumber some Hunt: Showdown videos in a playlist:

Well, now I have another issue. When we started playing Hunt: Showdown, I was publishing the videos a couple a day on Mondays, Wednesdays and Fridays. Putting them all out at once is a bit of a crass move as it floods subscribers with notifications, so spreading then out is the Done Thing.1

However we’re now above 150 videos, and even after adding weekends to the schedule that still takes us up to, umm, May.

What I’d like to do is go back and redo the schedule so that all pending videos use Saturdays and Sundays, and maybe think about doing three or four per day, which would bring us down to about 8/6 weeks’ worth. That is still a lot, quite frankly, but pushing up the frequency further would be detrimental.

Changing the scheduled publish date would be even more painful than renumbering because it requires more clicks, I’d have to keep track and figure out when the next one was supposed to go out, and there are more to do (120-odd).

So back to python! I have already written a schedule-determiner for automating the generation of the pre-upload json template, so I can reuse — read: from genjson import next_scheduled_date — that for this task.

The filtering logic is straightforward: ignore anything not a Hunt video, skip anything before a defined start date (ie videos already published). From there change the current ‘scheduled’ date for the next one from the new schedule.

For the current set of scheduled videos that are not already published, the schedule of 3 videos each 5 days (15 per week) gives:

Current date: 2020-04-06 17:30
New date : 2020-03-09 20:00

So we’ve saved a month! Plus the pending videos (~40) will be done in two and a half weeks instead of four.

From here it’s straightforward to rewrite the scheduled field and use shoogle as before to change the dates, this time setting publishAt under status. Note that privacyStatus needs to be explicitly set to private, even if it is already set! This avoids a “400 The request metadata specifies an invalid scheduled publishing time” error.

Another thing done quickly with python!


1: On the note of ‘Done Things’, the thing to do would be to upload fewer videos in the first place.

I’ve considered that, and if a video is truly mundane and missable, I will omit it. But as well as being fun/interesting videos of individual rounds, the playlist should serve as a demonstration of our progress as players. The Dead by Daylight playlist does this: we start with no idea what’s going on or how to play properly, and by the final video — somewhere north of 300 — we are pretty competent.

Categories
automation python timesavers video

Renumbering Ordered Videos in a YouTube Playlist with Python

Doing exactly what it says on the tin

I’ve been playing Hunt: Showdown with friends recently. With these kids of things I like to stream and record the footage of us playing so that others can share our enjoyment — highs and lows! — and so we can watch them back later.

The videos are compiled in a playlist on YouTube, in the order recorded. The tools that I’ve written to help automate the process of getting the videos from a file on a hard drive to a proper YouTube video include numbering.

I realised that I had missed out three videos, which would throw off the numbering. The easy options would be to:

  • add them to the end of the playlist; downside: the video number wouldn’t reflect the order and progression
  • insert them in the right place manually; downside: it would take a long time to manually renumber subsequent videos (about ~60)
  • write a script to do this for me

Guess which one I picked?

Interacting with YouTube programmatically comes in two min forms: APIs or a wrapper like shoogle. The latter is what I am familiar with, and has the benefit o’ being a braw Scottish word to boot!

The list of video files I’ve uploaded is in json format, which makes interaction a cinch. The list is loaded, anything not a Hunt: Showdown video is skipped*, a regex matches the video number, if it’s over a number (59) in this case the number in the title is increased by 4 (I also had a duplicate number in the list!).

This title is then set using shoogle. The API has certain things it expects, so I had to ‘update’ both the title and the categoryId, though the latter remained the same. You also have to tell the API which parts you are updating, which in this case is the snippet.

As an example, the json passed to shoogle might look like:

{ "body": {
    "id": <ID>,
    "snippet": {
        "title": "Golden Battle (Hunt: Showdown #103)",
        "categoryId": "20"
        },
    },
 "part": "snippet"
}

From here it’s a simple matter to invoke shoogle (I use subprocess) to update the video title on YouTube.

The one caveat I would mention is that you only get 10 000 API credits per day by default. Updating the video costs 50 units per update, plus the cost of the resource (for snippet this is 2), which works out to 192 videos per day, max.

Once the list has been updated, I dump out the new list.

Much quicker than doing it manually, and the videos all have the right number!

Categories
all posts games

Take 2 Claims ‘WZLJHRS’

Jack Howitzer as Jack Howitzer in ‘Jack Howitzer’

I played some GTA V: Online the other night — my three word review: ‘fun but clunky’ — and uploaded the footage of it as I usually do, leaving it as a draft to be later updated with my automation tools.

Later on I saw I had a notification on YouTube and thought “Ah! Someone’s subscribed, or commented, or similar”. Actually, I had a copyright claim from Take 2 Interactive for ‘WZLJHRS’. What?

“There are some visibility restrictions on your video. However, your channel isn’t affected. No one can view this video due to one or more of the Content ID claims below. WZLJHRS: Video cannot be seen or monetized; Blocked in all territories by Take 2 Interactive”

The just under two minute segment in question was a GTA teevee programme (‘Jack Howitzer’, a documentary/mockumentary about a washed up action movie actor) I watched while waiting for my friend to arrive at my office. It had some funny moments.

I am mindful of YouTube’s content ID system, and I mute game music pre-emptively having been bitten in the past by that. I didn’t suspect for a second that a fake TV show in a game would result in an entire video being blocked.

I will have to amend the video and reupload.

PS: WZLJHRS: WZL ? Weazel News network || JHRS ? Jack Howitzer show?

Categories
automation python timesavers video

Improving Generated JSON Template for YouTube Uploads

Further automation automation

On a few of my Europa Universalis series, I’ve used a quick little python script to do take care of some of the predictable elements of the series — tags, title and video number — and work out a schedule.

Having gone through the process of uploading a lot of Dead by Daylight videos in the past, and with a large and growing set of Hunt: Showdown videos building up it seems like a good time to start adapting that script.

There is a significant hidden assumption here: my video file names are in ISO 8601 format, so we can sort based on filename.

As the previous uses had been EUIV videos the parameters were coded in as variables. This is obviously undesirable for a general-purpose script, so we need some way of passing in the things we want. And since we’re outputting JSON, why not use JSON formatting for the parameters file too?

We look for a supplied directory and file pattern, and pass those to glob.glob to be os.path.join-ed to build the file list. We then use a sorted() copy of the list which will have the videos in the correct — see assumption — order for the playlist.

Iterating through this sorted list, we can set the basics that uploadytfootage expects.

The only ‘fancy’ work here is in figuring out the schedule dates. Quoting my own docstring:

"""Based on:
    - the current scheduled date
    - valid days [M,Tu,W,Th,F,Sa,Su]
    - valid times (eg [1600, 1745, 2100])
    return the next scheduled date"""

I debated whether to make this a generator; and in the end I avoided it for reasons I can’t quite remember.

First we look at hours: if there’s a valid time later in the current day, use that. If not, we set the new hours part to the earliest of the valid times.

Next, days: if there’s a valid day of the week in the current week, set it to the next one. If not, take the difference of the current day and the earliest valid day away from 7 and add that to get the new day. That one might need a bit of explaining:

Valid: Monday (1) || Current: Friday (5):
7 – (5 – 1) = 3.

Using 3 for the days component of the timedelta gives us the Monday following the current Friday. We can also set the hours and minutes component of the time in that timedelta object.

Then it’s simply a matter of returning the value of the current scheduled date plus the timedelta!

In addition, I skip changing the scheduled date for any video that has “part” in the filename; on the basis that if it’s just been split for length — such as a three hour EUIV video split into hour segments — the different parts should all go out on the same day.

Having all the dates in the schedule figured out and set automatically is a huge timesaver.

The JSON provided by genjson is valid as uploadytfootage goes; but the only things that really need done are setting a title (if the videos in the series have different titles; EUIV playlists tend not to, Hunt ones do), a description, a thumbnail title and a thumbnail frame time.

Doing those few things are much quicker than redoing the metadata for each and every video.

Categories
automation ocr python

Automating YouTube Uploads With OCR Part 9: Bringing it All Together

I love it when a plan comes together

We’ve been finding a way to automate YouTube uploads using tesseract to OCR frames from Deep Rock Galactic videos to extract metadata used in the video on YouTube.

We got to the stage where we have good, useful JSON output that our automated upload tool can work on. Job done? Well, yes- I could point the tool at it and let it work on that, but it would take quite a while. You see, to give a broad test base and plenty of ‘live-fire’ ammunition, I let a blacklog of a month’s videos build up.

Automating Metadata Updates

Why is that an issue for an automated tool? The YouTube API by default permits 10 000 units per day of access, and uploading a video costs 1600 units. That limits us to six videos per day max, or five once the costs of other API calls are factored in. So I’d rather upload the videos in the background using the web API, and let our automated tool set the metadata.

For that we need the videoIds reported by the API. My tool of choice to obtain those was shoogle. I wrapped it in a python script to get the playlistId of the uploads playlist, then grabbed the videoIds of the 100 latest videos, got the fileDetails of those to get the uploaded fileName… and matched that list to the filename of JSON entries.

So far so good.

Faster Thumbnails

But one of the personal touches that I like to do, and that will likely not be automated away is to pick a frame from the video for the thumbnail. So I need a way to quickly go through the videos, find a frame that would make a good thumbnail, and add that as a field to thumb for the correct video entry. I’ve used xdotool in the past to speed up some of the more repetitive parts of data entry (if you’ve used AutoHotKey for Windows, it’s similar to that in some ways).

I threw together a quick script to switch to the terminal with vim, go to the filename of current video in VLC (VLC can expose a JSON interface with current video metadata- the ones I’m interested in are the filename and the current seek position), create a thumb ? time entry with the current time and then switch back to VLC. That script can be assigned a key combo in Openbox, so the process is: find frame, hit hotkey, find frame in next video, hotkey, repeat.

Though the process is streamlined, finding a good frame in 47 videos isn’t the quickest! But the final result is worth it:

We have videos with full metadata, thumbnail and scheduled date/time set.

Glorious.

I included a video that failed OCR due to a missing loading screen (I hit record too late). There’s a handful of those- I found five while doing the thumbnails. I could do a bit of further work and get partial output from the loading/ending screen alone; or I could bit the bullet and do those ones manually, using it as a reminder to hit the record button at the right time!

Categories
automation programming python

Automating YouTube Uploads with OCR

Why?

I play games. Quite often I stream those games, and I also upload the footage to YouTube for my friends and subscribers to enjoy.

However, each session of a game is one video, so I end up with many videos. In fact, Jefe, I’d say I have a plethora of videos. Since they are different round of the same games, many of the videos have similar structure to their descriptions.

Lots of things being similar sounds like fertile ground for automation!

I have a system, described elsewhere, which uploads and publishes videos to YouTube based on metadata I write, which is vastly more convenient than doing it manually through the web interface, which is a bit clunky to work with when doing videos in any quantity.

But if the metadata is similar, what if we could automatically generate that?

Deep Rock Galactic

If you’re not familiar with Deep Rock Galactic, it’s a coop FPS game for up to four players that sees you going on missions in procedurally-generated caves on a fictional world to extract materials and kill aliens. It’s great fun, but don’t take my word for it, go watch some videos!

DRG has a loading screen that very helpfully includes all the information on it that is needed to generate the metadata for the YouTube video:

The loading screen. It has all the information about the mission and so the video. DRG devs, THANK YOU!

Let’s break down the elements:

Here we have the names of the brave dwarven miners. This lets me say who is in the video.

It also has the classes. I don’t use that information currently, but since it’s there I could.

This has the mission type (Point Extraction), and the generated name (Clouded Joy).

Lots going on here.

  1. Biome (location) of mission
  2. Potential minerals*
  3. Objectives
  4. Hazard level (difficulty)

* these are in pictograph format, but we can still work with that.

Descriptions

Let look at an example video for metadata and see how it maps up:

DRG Elimination: Rippled Outpost

BertieB and Costello brave the Glacial Strata to eliminate two Glyphid Dreadnoughts

It goes: <Game Name> <Mission Type>: <Mission Name>
<Players><Biome><Objectives>

And the rest of the metadata mentioned above is included in tags, but it could be put into the description just as easily.

All the elements are there, all we need to do is do a bit of image recognition on them. Fortunately python has bindings to such things, so as we’ve figured out where everything is, all that’s left to do is write the code- that’s the easy bit, right?