automation coding python video

Generating Text Captions for Shotcut

Making the video editing workload much lighter

Shotcut is a Free (GPLv3) cross-platform video editor. I’ve been using it a couple of times lately to put some simple clips together (like sorting the Take 2 copyright claim GTA Online video).

I figured I’d use it to take a clip of my friends and I getting schooled by someone with a bomb lance in Hunt: Showdown.

Actually, my first thought was to write a script to put a clip together using MELT — based on JSON, of course — but on reflection for these I wanted something a bit more refined.

So, enter Shotcut. One of the things I was keen to include were text-based captions. I’ve been including these in gifs (example) for a while now, and I think they work really well for video. They can be informative, and sometimes funny!

Text in Shotcut is doable natively via filters: text, HTML etc. But this felt awkward to me- I’d rather have something directly visible in the timeline which is easy to manipulate; and to add filters to itself if it comes to it.

So I decided… to write a script to generate images with these captions, based on — yup! — JSON. I quickly thew together a JSON file for the dialogue in clip I wanted to caption:

{ captions: [                                                                                                       
        [ 0, close by here],                                                                                        
        [ 0, other side of this wall],                                                                                      [ 1, yep yep yep],                                                                                          
        [ 2, That was a Sparks! :o],                                                                                
        [ 0, ohhhh fudge],                                                                                          
        [ 0, I die to this],                                                                                        
        [ 0, GADDAMMITTT],                                                                                          
        [1, what was that?],                                                                                        
        [0, bomblance :(],                                                                                          
        [1, where?],                                                                                                
        [2, he's with me],                                                                                                  [2, :(],                                                                                                    
        [0, you've got one bullet left],                                                                            
        [0, maybe on top if he's got a bomblance?],                                                                 
        [1, good idea],                                                                                             
        [0, is that not him at the gate?],                                                                          
        [1, dunno where he is],                                                                                             [2, he's on our bodies],                                                                                    
        [1, I know...],                                                                                             
        [1, WHAT?! *panicflee*],                                                                                    
        [1, this is a bit difficult],                                                                               
        [1, fuq! :(],                                                                                               
        [1, I should have run again],                                                                               
        [1, oh well],                                                                                               
        [0, "gg wp Flakel, you beat us o7"]                                                                         

Simple! The numbers refer to speakers; 0 is the first, 1 = 2nd, 2 = 3rd. I didn’t actually need to zero-index speakers, and in fact I can use text strings to denote who is speaking, but writing numbers is quicker if there’s twenty-five captions to do.

The script, which I will throw up on GitHub, goes through this and generates the caption for each item in the list. It has assigned colours for each ‘speaker’.

Due to familiarity, I was going to use imagemagick. But I originally used Pillow as I wanted to [re]gain a bit of familiarity with that. Once I had [re]acquainted myself with the few bits I needed it was relatively straightforward to generate a cropped image with the text appropriately sized, coloured and stroked; but I found myself wanting a full 1920×1080 frame as this made the Shotcut workflow much quicker since there was no need to set position if the image was the same size as the source video.

So I changed Pillow/PIL out for imagemagick and subprocess and redid the whole thing in a few minutes. The imagemagick version is significantly slower, but not so slow as to be intolerable even when wanting to tweak a couple of the captions.

I’m quite happy with how it turned out:

The ‘automatic’ text sizing could use a little tweak!

Lessons learned:

  • using something you’re familiar with is often easier than learning something new
  • PIL is faster than imagemagick for generating simple text on a transparent background
  • bomb lancers can be pretty deadly
automation python timesavers video

Rescheduling YouTube Videos using Python

More ‘exactly what it says on the tin’

A couple weeks ago, I had to renumber some Hunt: Showdown videos in a playlist:

Well, now I have another issue. When we started playing Hunt: Showdown, I was publishing the videos a couple a day on Mondays, Wednesdays and Fridays. Putting them all out at once is a bit of a crass move as it floods subscribers with notifications, so spreading then out is the Done Thing.1

However we’re now above 150 videos, and even after adding weekends to the schedule that still takes us up to, umm, May.

What I’d like to do is go back and redo the schedule so that all pending videos use Saturdays and Sundays, and maybe think about doing three or four per day, which would bring us down to about 8/6 weeks’ worth. That is still a lot, quite frankly, but pushing up the frequency further would be detrimental.

Changing the scheduled publish date would be even more painful than renumbering because it requires more clicks, I’d have to keep track and figure out when the next one was supposed to go out, and there are more to do (120-odd).

So back to python! I have already written a schedule-determiner for automating the generation of the pre-upload json template, so I can reuse — read: from genjson import next_scheduled_date — that for this task.

The filtering logic is straightforward: ignore anything not a Hunt video, skip anything before a defined start date (ie videos already published). From there change the current ‘scheduled’ date for the next one from the new schedule.

For the current set of scheduled videos that are not already published, the schedule of 3 videos each 5 days (15 per week) gives:

Current date: 2020-04-06 17:30
New date : 2020-03-09 20:00

So we’ve saved a month! Plus the pending videos (~40) will be done in two and a half weeks instead of four.

From here it’s straightforward to rewrite the scheduled field and use shoogle as before to change the dates, this time setting publishAt under status. Note that privacyStatus needs to be explicitly set to private, even if it is already set! This avoids a “400 The request metadata specifies an invalid scheduled publishing time” error.

Another thing done quickly with python!

1: On the note of ‘Done Things’, the thing to do would be to upload fewer videos in the first place.

I’ve considered that, and if a video is truly mundane and missable, I will omit it. But as well as being fun/interesting videos of individual rounds, the playlist should serve as a demonstration of our progress as players. The Dead by Daylight playlist does this: we start with no idea what’s going on or how to play properly, and by the final video — somewhere north of 300 — we are pretty competent.

automation python timesavers video

Renumbering Ordered Videos in a YouTube Playlist with Python

Doing exactly what it says on the tin

I’ve been playing Hunt: Showdown with friends recently. With these kids of things I like to stream and record the footage of us playing so that others can share our enjoyment — highs and lows! — and so we can watch them back later.

The videos are compiled in a playlist on YouTube, in the order recorded. The tools that I’ve written to help automate the process of getting the videos from a file on a hard drive to a proper YouTube video include numbering.

I realised that I had missed out three videos, which would throw off the numbering. The easy options would be to:

  • add them to the end of the playlist; downside: the video number wouldn’t reflect the order and progression
  • insert them in the right place manually; downside: it would take a long time to manually renumber subsequent videos (about ~60)
  • write a script to do this for me

Guess which one I picked?

Interacting with YouTube programmatically comes in two min forms: APIs or a wrapper like shoogle. The latter is what I am familiar with, and has the benefit o’ being a braw Scottish word to boot!

The list of video files I’ve uploaded is in json format, which makes interaction a cinch. The list is loaded, anything not a Hunt: Showdown video is skipped*, a regex matches the video number, if it’s over a number (59) in this case the number in the title is increased by 4 (I also had a duplicate number in the list!).

This title is then set using shoogle. The API has certain things it expects, so I had to ‘update’ both the title and the categoryId, though the latter remained the same. You also have to tell the API which parts you are updating, which in this case is the snippet.

As an example, the json passed to shoogle might look like:

{ "body": {
    "id": <ID>,
    "snippet": {
        "title": "Golden Battle (Hunt: Showdown #103)",
        "categoryId": "20"
 "part": "snippet"

From here it’s a simple matter to invoke shoogle (I use subprocess) to update the video title on YouTube.

The one caveat I would mention is that you only get 10 000 API credits per day by default. Updating the video costs 50 units per update, plus the cost of the resource (for snippet this is 2), which works out to 192 videos per day, max.

Once the list has been updated, I dump out the new list.

Much quicker than doing it manually, and the videos all have the right number!

automation python timesavers video

Improving Generated JSON Template for YouTube Uploads

Further automation automation

On a few of my Europa Universalis series, I’ve used a quick little python script to do take care of some of the predictable elements of the series — tags, title and video number — and work out a schedule.

Having gone through the process of uploading a lot of Dead by Daylight videos in the past, and with a large and growing set of Hunt: Showdown videos building up it seems like a good time to start adapting that script.

There is a significant hidden assumption here: my video file names are in ISO 8601 format, so we can sort based on filename.

As the previous uses had been EUIV videos the parameters were coded in as variables. This is obviously undesirable for a general-purpose script, so we need some way of passing in the things we want. And since we’re outputting JSON, why not use JSON formatting for the parameters file too?

We look for a supplied directory and file pattern, and pass those to glob.glob to be os.path.join-ed to build the file list. We then use a sorted() copy of the list which will have the videos in the correct — see assumption — order for the playlist.

Iterating through this sorted list, we can set the basics that uploadytfootage expects.

The only ‘fancy’ work here is in figuring out the schedule dates. Quoting my own docstring:

"""Based on:
    - the current scheduled date
    - valid days [M,Tu,W,Th,F,Sa,Su]
    - valid times (eg [1600, 1745, 2100])
    return the next scheduled date"""

I debated whether to make this a generator; and in the end I avoided it for reasons I can’t quite remember.

First we look at hours: if there’s a valid time later in the current day, use that. If not, we set the new hours part to the earliest of the valid times.

Next, days: if there’s a valid day of the week in the current week, set it to the next one. If not, take the difference of the current day and the earliest valid day away from 7 and add that to get the new day. That one might need a bit of explaining:

Valid: Monday (1) || Current: Friday (5):
7 – (5 – 1) = 3.

Using 3 for the days component of the timedelta gives us the Monday following the current Friday. We can also set the hours and minutes component of the time in that timedelta object.

Then it’s simply a matter of returning the value of the current scheduled date plus the timedelta!

In addition, I skip changing the scheduled date for any video that has “part” in the filename; on the basis that if it’s just been split for length — such as a three hour EUIV video split into hour segments — the different parts should all go out on the same day.

Having all the dates in the schedule figured out and set automatically is a huge timesaver.

The JSON provided by genjson is valid as uploadytfootage goes; but the only things that really need done are setting a title (if the videos in the series have different titles; EUIV playlists tend not to, Hunt ones do), a description, a thumbnail title and a thumbnail frame time.

Doing those few things are much quicker than redoing the metadata for each and every video.

coding timesavers video

Quick Hacks: A Script to Extract a Single Image/Frame From Video

Long ago, I posted the simple way to get a frame of a video using ffmpeg. I’ve been using that technique for a long time.

It can be a bit unwieldy for iteratively finding a specific frame, as when using a terminal you have to move the cursor to the time specification. So I wrote a very small wrapper script to put the time part at or towards the end:

# - single frame

USAGE=" infile timecode [outfile]"

if [ "$#" == "0" ]; then
        echo "$USAGE"
        exit 1

if [ -e "$1" ]; then
        echo "file not found: $1"
        exit 1

if [ ! -z "$2" ]; then
        echo "Need timecode!"
        exit 1

# if we have a filename write to that, else imagemagick display

if [ ! -z "$3" ]; then
        echo "ffmpeg -i \"$video\" -ss $time  -vframes 1 -f image2 \"$3\""
        ffmpeg -loglevel quiet -hide_banner -ss $time -i "$video" -vframes 1 -f image2 "$3"
        echo "ffmpeg -i \"$video\" -ss $3  -vframes 1 -f image2 - | display"
        ffmpeg -hide_banner -loglevel quiet -ss $time  -i "$video" -vframes 1 -f image2 - | display

Most of that is usage explanation, but broadly it has two modes:

  • display an image ( video time)
  • write an image ( video time image)

It’s more convenient to use it, hit ? and amend the time than to move the cursor into the depth of an ffmpeg command.


Rescuing a Truncated Recording Using ffmpeg

tl;dr: Well, it’s cheating because it’s not ‘rescuing’ what isn’t recorded, just retrieving and appending

Last night I played a few rounds of some cooperative gameplay with some good friends, and as usual I recorded the session for posterity.

Hans Volter's on fire and about to die
“Oh man, oh god!”, Hans is about to croak.

However, when looking over the file created, neither VLC nor ffmpeg could determine a duration. I used ffmpeg to create a new file (ffmpeg -i old.mkv -c copy new.mkv), and when it got to about the 1 hour 50 minute mark in the stream copy, it printed the following error:

invalid return value 0 for stream protocol
[matroska,webm @ 0x55eb60b609c0] Read error
Invalid return value 0 for stream protocol

It’s possible that I ran out of space during recording, so it behoves me to make sure my disks are more clear — hopefully the process of migrating to a combined fileserver will help there! — before I start recording.

Fortunately, I also stream to Twitch and so I could create a ‘highlight’ of the latter part of gameplay- about 45 minutes’ worth. The quality isn’t as good (max bitrate 3500kbit), but it’s better than nothing. So that it can be seamlessly appended, I decided to upscale the size from 720p to 1080p using ffmpeg:

$ ffmpeg -i ~/downloads/211006613-69819912-48eb80ad-009f-42d5-83aa-f68dda7b7612.mp4 -vf scale=1920x1080:flags=lanczos
-c:a copy -crf 20 -preset slow ~/mounts/storage/video/2017-12-19\ 22-04-59b.mkv

frame=105361 fps= 24 q=28.0 size= 2402802kB time=00:29:16.20 bitrate=11208.1kbits/s speed=0.408x

(Conversion in progress! Using -vf scale=1920x1080:flags=lanczos to scale and -crf 20 to ensure quality stays reasonable)

After that, it should be a simple matter of using the concat demuxer, assuming the files are similar enough (both are pixel format yuvj420p)– I previously had issues when trying to create an auto-highlighter/clip creator that minimised re-encoding.

For posterity!

all posts video

An Old Video (FEAR – Stolen Kills)

I’ve been posting a fair number of videos to another blog for the folk I play games with (including the other MIA author of posts on this blog Kenny). It’s a good way to relive some moments and also a way for me to practice my cutting and splicing skills.

Long story short, I encoded an updated version of a video I created about three and a half years ago. The old one was small resolution and tiny bitrate. This version is full (source) resolution and a relatively insane bitrate.

FEAR – Stolen Kills (pop-out player if you have JS enabled, non pop-out version here)

Synopsis: I shoot Kenny in the back (with friendly fire on) and I’m rewarded with two kills. Obviously a sounds strategy!