Archive for the ‘computing’ Category

Lost network connection with Ubuntu upgrade?

April 17, 2017

Did you upgrade your Ubuntu installation to 16.10 or 17.04 recently?

Did you have an Ethernet connection to the outside world?

Are you now not connected?

If you can answer yes to all these questions, this blog post may be for you. I have found this will fix it. This may not be the best solution however:

sudo rename /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf /usr/lib/NetworkManager/conf.d/_10-globally-managed-devices.conf_backup
sudo service network-manager restart

If you’re not bothered about keeping a copy of the file, just ‘rm’ it.

BBC iplayer “loophole”.

September 1, 2016

Everyone has their $0.02, and so do I.

Since the fanfare-like announcements in recent weeks I was mildly curious. Today, as I type, I am downloading Series 2, Episode 5 of the BBC documentary series “Inside The Factory”.

At no point during this process have I needed to prove I am in possession of a valid TV Licence. I didn’t bother clicking on the part of the page where it says “I have a TV Licence”, I just needed the pid (Programme ID) from the URL bar.

Citrus Lime of Ulverston

December 30, 2015

Please stop sending password reminder email messages which contain the user’s password. Email messages are liable to be transmitted over non-encrypted connections, which may allow a third-party to intercept the message.

You claim “Not only are we offering a cutting edge Retail System, but the expertise and knowledge to drive all our customers forward to develop and grow their businesses.

I say: Risking the confidentiality of your client’s customers is a poor way to do business.

You need to change the system so that a temporary password is set for the account, and the user needs a second piece of information (date of birth, postcode etc) to reset the account password.

Please let me know when you’ve sorted this out.

Unpacking D-Cinema content.

April 4, 2014

This entry is intended to be read after my earlier entry on a similar subject:


Having provided a (relatively) simple system for putting your content in to DCP format, I now present a simple system for recovering it. When considering publishing this information I have tried to anticipate some of the outcomes. I have discussed it with people whose judgement I trust. I use no exploit, or “hack” or “bypass” here. I trust that you will only use this information to process content you have the rights to, or only with the full authority of the copyright holder.
The DCI specification provides a means to protect content against unauthorised access, and against tampering. The cinema industry uses it every day. There is no excuse not to use content protection in the cinema industry, particularly when key management is now so streamlined and automated.


If you did not install the digital_cinema_tools package:
then do it now because you will need it for the asdcp-test command

Other requirements are:
(1) OpenShot
(2) ImageMagick
(3) mencoder
(4) A working knowledge of how a DCI package is organised
(5) Lots of disk space
(6) A supply of tea, or your preferred beverage(s)

You might want to turn off image thumbnail creation, as this can easily generate in excess of 6000 files in your working folder. At various points in this tutorial you will find yourself looking for various bits of information about the content such as frame rate, aspect ratio e.t.c. Most of this is held in the Control Play List (CPL) file. The ASSETMAP will define which file is the CPL. The contents are held in XML, so open the ASSETMAP, CPL etc in a web browser.

As the DCI Specification is tightly defined, this procedure ought to work on any JPEG-2000 encoded Digital Cinema Package (DCP) with these conditions:
(a) Not encrypted (no KDM required)
(b) Not stereo (2D content)
(c) Consists of only one reel – most shorts, trailers, adverts, public service announcements etc.

Extracting the picture stream, and conversion from CIE 1931.

First, read the ASSETMAP file. This is found within the folder which holds all the files which (together) comprise the DCP. The ASSETMAP file should tell you where to find the PacKing List (PKL). Search for “<PackingList>” and look a few lines down for the name of the actual PKL file.
Open the PKL file, and look for “asdcpKind=Picture”. That will give you the name of the MXF file which holds the picture information. Also there should be a pointer to the MXF file for the sound, make a note of that for later.

For the purposes of this procedure, I will assume you have determined that the picture file is called “MXF-video.mxf”. Open a command window, cd to the correct folder and issue this command (changing the name of the MXF file to suit):

asdcp-test -x frame MXF-video.mxf

What will happen is asdcp-test will create files of the names frame000000.j2k, frame000001.j2k, frame000002.j2k e.t.c. One file per frame of the DCP. These will be in JPEG-2000 format.

You may be able to import these files directly in to your video editing software. I will provide means to convert them in to other formats, which are more readily understood.

What you should consider at this point is that you may lose some colour information. DCI content uses the CIE 1931 colour space model, and gives a maximum of 12 bits per colour component, therefore 36 bits per pixel. In theory you could convert these JPEG-2000 images to TIFF format, and preserve all that colour resolution, but not all applications understand 48 bits per pixel TIFF files.

To be fair, I’m not sure how much it matters as my display device is 10 bits per channel at best. If you’re using a notebook, you may well only have 6 bits per channel.

To convert to PNG files use this (with a tiny loss of colour accuracy):

mogrify -format png -alpha Off -resize 999x540 -gamma 0.384615 -set colorspace XYZ -colorspace sRGB -depth 8 *.j2c

(a) The output images are resized, feel free to adjust the target size to suit your requirements. I’ve simply divided by 4 to get these figures.
(b) Notice the use of the unusual gamma, because in D-Cinema gamma is 2.6 at projection.
(c) Use “-depth 16” and “format tiff” if you want to preserve all colour resolution. Worth doing for selected stills, not worth it for transcoding to video.
(d) This will take a long time, go and make a cup of tea. Other beverages may be available.

At this point you can pick out your favourite still frame.

If you intend to proceed all the way to producing a consumer-friendly video of the content, then read on.

Sorting out the sound.

The sound is not so hard to deal with, as the DCI Specification requires uncompressed sound. Common media player applications such as VLC can play audio held in the MXF container format without any sort of conversion. For simplicity’s sake, and because this post is mainly about the picture, I suggest converting the sound to a 2 channel WAV file like this:

mplayer MXF-audio.mxf -ao pcm:file=audio.wav,fast -channels 6 -af pan=2:0.4:0:0:0.4:0.2:0:0:0.2:0.3:0.3:0.1:0.1

This command mixes the six channel sound down to two channels in a standard RIFF format WAV file. The pan option tells mplayer how to mix each input channel to make two output channels. (Source thank you, it sounds slightly better than the example in the manual)

Encoding the video and sound.

You can use some magic command line incantations for this, but it is easier to import the image sequence in to OpenShot along with the 2 channel sound file.

Within OpenShot, create a profile which matches your chosen picture resolution and frame rate. Refer back to the mogrify command you used earlier.

(1) Frame rate is likely to be 24fps, as not all DCinema kit fully supports all valid frame rates. You will find it in the CPL file.
(2) The aspect ratio is likely to be 1.85, but look in the CPL file to confirm. Openshot will accept aspect ratios as a fraction, so you can use 185/100 or the simplest version as 37/20.
(3) The pixel aspect is “1/1”.

There is something which might trip you up. Check the CPL for the “<ReelList>” section. For the picture and sound there might be non-zero “<EntryPoint>” values. These values describe an offset within the video and audio, to show where playback should commence. Using the frame rate and offset (number of frames), calculate the offset between picture and sound. If both of these are zero, then it’s easier later on in OpenShot.

I did not set out to write an OpenShot tutorial, but there is a feature of OpenShot which is not well known. If you drag a file named similar to “image000000.png” to the Project File panel, OpenShot will recognise it as the first frame of a set of sequentially numbered frames. See this tutorial for the details. Add the sound file to the project in the same way.

Put the video on track 2 (upper), and sound on track 1 (lower).

If you have non-zero <EntryPoint> values: With reference to the offset you calculated earlier, move the video or sound left or right as required. If the picture needs to be ahead of sound, you can right-click and enter the offset with reference to the project time line.

Play it through to confirm correct synchronisation.

Use the Razor tool to remove the excess picture/sound from the beginning and end of the project as required. If you’re left with a blank space at the zero line of the project, move both tracks so they commence at zero on the timeline.

Export the project as (rendered) Video, check the profile in use is correct (matches resolution and frame rate) and choose sensible values for container format, video codec, audio codec and frame rate. Some versions of OpenShot (currently) seem to have a problem in saving the finished file to any other folder than /home, so don’t panic if the file is not where you expected it to be.

Tidying up.

(Use these commands with care)

rm *.j2c
rm *.png
rm *.tiff
rm audio.wav

Final words and bits that don’t fit anywhere else.

There was a discussion about adding support for converting XYZ colours to RGB to ffmpeg. I’m no stranger to compiling code from source, but I’ve not tried this method.
As with my previous post on this subject, I extend my thanks to the same four people – for the same reasons.
Actually you can use this with 3D material, start with asdcptest -3 -x frame … and it will output all left eye and right eye frames. I’ll leave it as an exercise for the reader about further processing of the data.

Printing an image larger than the paper size.

February 6, 2014

This another of those “for personal reference posts”, although it might be useful for others. Like many of my posts on computing, this is relevant to Debian-derived Linux i.e. Ubuntu.

If you want to print a banner, or poster, and only have a printer which can print on (say) A4 paper then there is a simple solution. Save the image at the correct size in EPS format. I used Inkscape for this, and I had the paper size set to 1100mm by 550mm.

Check that poster is installed (if not, it is available in the standard repositories).

The command to print my design at 1:1 scale, using A4 size pages:

poster -v -mA4 -s1 InputFile.eps >

The switches specify verbose output (-v), use A4 size paper (-mA4) and that the scaling factor is 1:1 (-s1).

The output file can be printed from Evince, or your favourite Postcript-aware application. It is worth reading the manual for poster too.

Rendering images in 3D for your 3D TV with POV-ray.

November 19, 2013

Having a little bit of spare time this week, and being aware of the broadcast of “The Day Of The Doctor” in 3D, I have turned my attention to making sure the 3D function of my TV is working properly.

The problem is content, or how to get it without paying for it. A solution would be to make it from resources already available for free.

There are many Ray-tracing software packages available, one of those is POV-Ray.
I’m using version 3.6, I know version 3.7 is now available but there might be unanticipated features. In common with some of my other compute-oriented posts I’m using Ubuntu Linux. Pre-compiled windows binaries of POV-Ray, Imagemagick and EXIFTool are available.
I acknowledge the hard work and expertise of Friedrich A. Lohmüller with his excellent pages on the subject of rendering stereo pairs with POV-Ray here.

I have tested the output with my Panasonic TX-P42UT50B TV and the corresponding 3D glasses for it.

After installing POV-Ray, and checking you have the other software Imagemagick and EXIFTool available, you need to choose a POV-Ray scene file to render. For my test I used the woodbox.pov scene from the folder /scenes/advanced.

You will need to modify the scene file, so copy it and insert the lines which begin “// stereo view:“. The the scene file will have a camera declaration already, so make a note of the location and look_at co-ordinates. Copy these in to the stereo camera section, then you can delete the now-redundant (non-stereo) camera declaration. Save the scene file.

On Friedrich A. Lohmüller’s page there is a short .ini file. You need to copy this file to your work folder. You must use the same name for your scene file, and the .ini file. This .ini file automates the rendering of the left and right eye images.

At this point it is a good idea to do a low-resolution test render of your image. If your scene file is called test.pov and you should have a file called test.ini in the same folder. Run this command:

povray test.ini +Itest.pov +Otest.tga +W480 +H270 +FT

The result should be two images. You can view them in your favourite image viewer, but if you want to view them in 3D there is a simple way to do that. Run this command:

montage test1.tga test2.tga -geometry +0+0 stereo.png

To make a side-by-side image suitable for cross-eyed viewing.

You might find the camera is too close, or too far away from your scene. You can adjust the camera location by changing it’s position as defined on the line:

#declare Camera_Position=<x,y,x>;

What you’re actually changing is the centre-point between the two viewing positions, left and right. Re-render the preview until you’re happy with it. When you are ready to move on to the next stage do this:

povray test.ini +Itest.pov +Otest.tga +W1920 +H1080 +FT +A0.3

It’s probably a good idea to make yourself a cup of tea at this point. When the rendering is complete, the next task is to make the images viewable on your TV. Convert them to the JPEG format:

convert test1.tga -quality 85 test_R.jpg
convert test2.tga -quality 85 test_L.jpg

This last step is to convince the TV that the images were created at the same time, use EXIFTool to make the Create Date tag on both images identical. Note that the content of the tag does not need to be accurate, just match on both images:

exiftool -CreateDate=201311191000 test_L.jpg -overwrite_original
exiftool -CreateDate=201311191000 test_R.jpg -overwrite_original

Copy the images to an SD card, and insert that in to your TV. With my TV, you need to tell it that you want to view a stereo pair. Having started the photo viewer on the TV, press Option and it’s the second menu entry. Choose the left image first with the red button. The letter L appears before R in the alphabet, and the TV always sorts the images in the same way.

Conforming and converting video in to Digital Cinema Package format.

March 29, 2013

This might be quite a long read, make yourself comfy.

When I started using an action camera to record my biking, someone asked me if it was possible to play the video back at the local cinema. I will call this guy Dave, because that’s his name but it also has a degree of anonymity because it’s a popular name. At time I was working at a local cinema, which had recently changed all the film projectors for digital projectors. I doubt Dave was being completely serious about the idea, but it set me thinking about the possibilities.

I present a work-flow to convert the output of my camera (CamSports HD-S 720p). Reasonably you could use any digital video camera if you make the appropriate adjustments to the conforming instructions.

I use Ubuntu as my “mostly-used” operating system, this is tested on version 12.10. It should work on most, if not all, Debian-type distributions. An advantage of using linux for this task is that you don’t need to be worried about file-size or filename limitations, because you can save directly on to an EXT3 formatted disc, with an MBR type partition table.
You need lots of disk space, as you will be converting your video to a sequence of still frames, and doing conversions on that. The sound will grow in size too, because D-Cinema does not understand compressed sound (AC3, DTS and SDDS are so ‘old hat’). Consider having your OS installed on an SSD, with your workspace on a big “traditional” hard disk.

Installing the software, and general preparation.

Visit and download the corresponding package for your system. Install it manually:

sudo dpkg -i opendcp_0.27.1_xubuntu1210_i386.deb

If you get errors about other packages not being installed, they can usually be fixed with this command:

apt-get -f install

There are some other packages you will need:

sudo apt-get install openjpeg-tools
sudo apt-get install imagemagick
sudo apt-get install sox
sudo apt-get install ffmpeg

I am not sure if OpenDCP needs openjpeg-tools, but it’s useful to be able to sample check your JPEG2000 encoded frames.

Some other preparation, make yourself a work folder and some sub-folders within:

mkdir DCP_Project
cd DCP_Project
mkdir audio
mkdir tiff
mkdir j2k
mkdir mxf
mkdir dcp

Copy your video file in to the DCP_Project folder.

Conforming your source file.

This is where you will adjust your video and audio, to make it fit within the rigid formats D-Cinema uses. In this example I am using the file “20120624.mp4”. The first step is to use ffmpeg to convert the video in to a series of individual frames. Notice that I sneak in a frame-rate conversion here too:

ffmpeg -y -i 20120624.mp4 -an -r 24 -pix_fmt rgb24 -vcodec tiff tiff/%08d.tiff

These frames are all the wrong size, and need tiny black bars left and right. The target is a vanilla 2k, JPEG2000, 2D, 24fps DCP.

cd tiff
mogrify -resize 1920x1080 *.tiff
mogrify -border 39x0 -bordercolor Black *.tiff
cd ..

The frames should all be 1998×1080 pixels which corresponds to the “flat” DCI format.

The next job is the audio. My camera records one sound channel only, and it does that badly. For simplicity’s sake I will make that channel appear on the centre (screen) channel in the cinema, and supply silence for the other 5 channels:

ffmpeg -y -i 20120624.mp4 -acodec pcm_s24le -ar 48000 audio/audio.wav
cd audio
sox audio.wav left.wav remix 0
sox audio.wav right.wav remix 0
sox audio.wav center.wav remix 1
sox audio.wav sub.wav remix 0
sox audio.wav surrleft.wav remix 0
sox audio.wav surrright.wav remix 0
cd ..

Converting and creating the DCP.

Start the OpenDCP application and select the JPEG2000 tab. The default options should be correct (OpenJPEG, Cinema 2K, 24 frame rate, 125Mb/s bandwidth, sRGB source, X’Y’Z’ Transform, DPX Logarithmic does not matter for tiff input, DCI Resize: None).
Set the input directory to your “tiff” folder, and the output folder to your “j2k” folder. Click on Convert, and wait.

Optional: Check one or two sample images from the “j2k” folder, by opening them in GIMP.

In OpenDCP select the MXF tab next. Convert the sequence of JPEG2000 images (“j2k” folder) in to an MXF file. The options should be JPEG2000, SMPTE, frame rate 24. Leave Stereoscopic and Slideshow unselected. Set the picture input to your “j2k” folder, and the output should have filename “video.mxf” in the “mxf” folder.

Staying with the MXF tab, change the type option to “WAV” and select input type “Mono” and output type “5.1”. There are now six input files required (in order):
Left Surround
Right Surround
Match those up with the six files created with sox earlier. Set the output file to be “audio.mxf” to be saved in the “mxf” folder, and create the MXF file.

Optional: Check listen to the audio.mxf using VLC.

This project does not have subtitles, so skip over that step and go to the DCP tab.

Use the Title Generator, as the DCI specification has a tightly controlled title naming system. This project is a “short” so that would be a good type or kind for setting up the Composition Parameters.
Adding a XML Digital Signature is a good idea, it detects if your DCP has been corrupted in transit. You might find the process of creating that signature seems to hang at 99%. This is a known issue with OpenDCP right now.
As this project only has one “reel” just fill in the Picture source (“video.mxf”) and Sound source (“audio.mxf”). Leave the Subtitle source blank. You might notice that frame counts are not the same, reduce the largest to make it the same as the smallest. This depends on your source material, and may also depend on the phase of the moon.
Create your DCP, and you’re nearly finished.

As you are using linux, you can use gparted to create your distribution disc (MBR partition table, EXT3 format) and copy over your completed Digital Cinema Package. A good thing is to create a folder with the same name as the package, and put the six files in that folder. In my example I created a DCP called “20120624_SHR_F_EN-XX_UK-U_51_2K_20130324_OPE_2D_OV” so I put all the files in a folder of that name.

Unmount the drive, and send it to your cinema.

Optional verification step.

You could follow the instructions at
to check what you have created.

The general form of the command is to point it at the folder containing your DCP. Using my example:

dcp_inspect dcp

This gives the following output:

 dcp_inspect v1.2013.03.22: ["dcp"]
 Found 1 Assetmap candidate: ["ASSETMAP.xml"]
 AM 118cd216-6685-4f58-98ec-12f8dfb89bb3: Schema check: OK: dcp/ASSETMAP.xml

 Found 1 Assetmap
 Assetmap 118cd216-6685-4f58-98ec-12f8dfb89bb3: ASSETMAP.xml
 Assetmap 118cd216-6685-4f58-98ec-12f8dfb89bb3 lists 4 assets:
 e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4_pkl.xml
 ea68ce2b-a73a-45d3-ba8b-3a6059de7351: ea68ce2b-a73a-45d3-ba8b-3a6059de7351_cpl.xml
 c07f1811-7c8b-4d79-8920-d465c967bf02: video.mxf
 f05fef5c-86d4-4283-b19e-8dfa54c6bf2e: audio.mxf
 Assetmap 118cd216-6685-4f58-98ec-12f8dfb89bb3 lists 1 PKL:
 PKL file present: e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: dcp/e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4_pkl.xml

 Found 1 Package
 PKL file present: e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: dcp/e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4_pkl.xml

 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: dcp/e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4_pkl.xml
 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: Schema check: OK
 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: Signature check: OK
 Signer:        /
 Signer issuer: /
 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4 lists 3 assets
 c07f1811-7c8b-4d79-8920-d465c967bf02: application/mxf: video.mxf
 c07f1811-7c8b-4d79-8920-d465c967bf02: Checking hash value: OK                                                       
 f05fef5c-86d4-4283-b19e-8dfa54c6bf2e: application/mxf: audio.mxf
 f05fef5c-86d4-4283-b19e-8dfa54c6bf2e: Checking hash value: OK                                                       
 ea68ce2b-a73a-45d3-ba8b-3a6059de7351: text/xml: ea68ce2b-a73a-45d3-ba8b-3a6059de7351_cpl.xml
 ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Checking hash value: OK                                                       
 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4: Package size: 551.6 MB
 PKL e508a2c6-b28b-4ff8-8b7d-7bbf0bf152b4 lists 1 composition
 CPL file present: ea68ce2b-a73a-45d3-ba8b-3a6059de7351: dcp/ea68ce2b-a73a-45d3-ba8b-3a6059de7351_cpl.xml

 Found 1 Composition
 CPL file present: ea68ce2b-a73a-45d3-ba8b-3a6059de7351: dcp/ea68ce2b-a73a-45d3-ba8b-3a6059de7351_cpl.xml

 CPL ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Schema check: OK
 CPL ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Signature check: OK
 Signer:           /
 Signer issuer:    /
 CPL Id:           ea68ce2b-a73a-45d3-ba8b-3a6059de7351
 CPL file:         dcp/ea68ce2b-a73a-45d3-ba8b-3a6059de7351_cpl.xml
 CPL type:         SMPTE (
 ContentTitleText: 20120624_SHR_F_EN-XX_UK-U_51_2K_20130324_OPE_2D_OV
     Language audio and subtitles: EN-XX
 AnnotationText:   20120624
 ContentKind:      short
 IssueDate:        2013-03-24T05:59:50+00:00 (Sunday Mar 24 2013 05:59)
 Issuer:           <Not present>
 Creator:          OpenDCP 0.27.1
 Number of Reels:  1
 Reel 1:
   1031  00:00:42+23 @ 24/1  c07f1811-7c8b-4d79-8920-d465c967bf02  MainPicture    (SMPTE, 00:00:43+01, unencrypted, 1998x1080, pictures)
   1031  00:00:42+23 @ 24/1  f05fef5c-86d4-4283-b19e-8dfa54c6bf2e  MainSound    (SMPTE, 00:00:42+23, unencrypted, 6 channels, 48 kHz, 24 bps, audio)
 Total duration:
   1031  00:00:42+23 @ 24/1
 Composition kind: 2D/Monoscopic
 Composition type: SMPTE
 Composition complete

 Hint: CPL ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Naming convention: Expected number of parts present in "20120624_SHR_F_EN-XX_UK-U_51_2K_20130324_OPE_2D_OV"
 Hint: CPL ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Naming convention: 11 parts matching: {:film_title=>"20120624", :content_type=>"SHR", :aspect_ratio=>"F", :language=>"EN-XX", :territory_rating=>"UK-U", :audio_type=>"51", :resolution=>"2K", :studio=>"OPE", :date=>"20130324", :facility=>"2D", :package_type=>"OV"}
 Hint: CPL ea68ce2b-a73a-45d3-ba8b-3a6059de7351: Naming convention: 0 parts missing: {}
 Info: Inspected: /home/alexander/Videos/dcptest/dcp
 Info: Found 1 Package with total size 551.6 MB
 Info: Found 1 Assetmap, 1 Package (1 signed/1 verified), 1 Composition (1 signed/1 verified)
 Info: 0 Errors, 3 Hints

The important thing is “0 Errors”.

Obviously I’m not in a position to actually test this on a real D-Cinema system…

There are people who have had success with these tools, some examples:

Finally, a big thank-you to other people like Diane Edwards, Daniel Backhouse, Andrew Bailey and Andrew Smith. You all provided me with the necessary impetus to learn the DCI specification to a much deeper understanding compared to anyone else I met during my time with Apollo Cinemas.