OpenGL without X11 on Raspberry Pi 4 (Part 2)

The next step on my OpenGL without X11 journey was to get the samples from both the “OpenGL ES 2.0 Programming Guide” and “OpenGL ES 3.0 Programming Guide” working. This has now been achieved. The rest of this blog will simply contain the build instructions to get the samples building.

2.0

git clone https://github.com/DrGeoff/opengles-book-samples.git
cd opengles-book-samples/LinuxDRM
make
# TADA!
find . -type f -executable    # Show what got built

3.0

git clone https://github.com/DrGeoff/opengles3-book.git
cd opengles3-book
mkdir build
cd !$
cmake -DUseDRM=1 -DCMAKE_BUILD_TYPE=Debug ..
make
# TADA!
find . -type f -executable   # Show what got built

Tagged with: , , ,
Posted in Uncategorized

pvtrace (dynamic stack trace visualisation)

Understanding the actual code flow of an application can be tricky at times, particularly in the presence of function pointers. A typical methodology for following code flow is to step through the program in gdb, but that can be tediously slow. What if you could record every function entry and exit?

By utilizing the gcc compiler option -finstrument-functions a dynamic function address trace can be recorded. Once you have that dynamic stack trace, the pvtrace utility written by M. Tim Jones can convert that recorded trace into a graphviz dot graph that can then be visualised.

An article containing full details of this procedure and how it works behind the scenes was written by the pvtrace source code author M. Tim Jones in 2005. This article was once available on the IBM developer works website but now both the article and source code are a little hard to come by. Thanks to the wayback machine I’ve been able to acquire both the article and the source code and I’ve created a github repository (https://github.com/DrGeoff/pvtrace.git) to make them more easily accessible.

Tagged with: , , ,
Posted in Uncategorized

OpenGL without X on Raspberry Pi 4 (Part 1)

Introduction

Many people wanting to use OpenGL on a Raspberry Pi will be happy to boot into the graphical desktop and use OpenGL inside an X11 window. If you are new to graphics programming then these are not the blogs you are looking for.

If you have previously used RPi0-3 without X11 then you may have used DispmanX to obtain a Native Window. Using a variety of intertube sources, it now seems that with the advent of the RPi4, the use of the DispmanX library has moved from discouraged to the wrong way. So what do you use? The answer seems to be, use the Linux Direct Rendering Manager (DRM).

I should fess up and let you know that I have never used libDRM before so this blog is an account of my stumblings around in the dark.

KMSCube

The canonical OpenGL without X example code is given in a program called KMSCube. Source code is available at https://github.com/robclark/kmscube but you can install a binary version using

sudo apt install kmscube

Your first instinct will be to try to run

/usr/bin/kmscube

which will fail with

drmModeGetResources failed: Invalid argument
failed to initialize legacy DRM

What is going on here is that by default kmscube tries to use /dev/dri/card0 as the default, whereas the hardware we want to use is card1.

Pi0-3 you only have the vc4 driver, therefore render is via /dev/dri/card0. Pi4 has both vc4 for render, and v3d for 3D. You can probe the devices for their capabilities – only one should acknowledge that it has DRIVER_RENDER or DRIVER_MODESET capabilities.

https://www.raspberrypi.org/forums/viewtopic.php?t=247540#p1511538

No worries, kmscube has your back,

/usr/bin/kmscube -D /dev/dri/card1

but boom!

<snip>
OpenGL ES 2.x information:
  version: "OpenGL ES 3.1 Mesa 19.3.2"
  shading language version: "OpenGL ES GLSL ES 3.10"
  vendor: "VMware, Inc."
  renderer: "llvmpipe (LLVM 9.0.1, 128 bits)"
<snip>
failed to set mode: Invalid argument

One of the unexpected outputs in the above is that the renderer is the software renderer, llvmpipe, rather than the Raspberry Pi V3D engine. I also note that it failed to “set mode” which makes me think “permissions!” So let’s take a look at the permissions of card1 and our own groups:

$ ls -al /dev/dri/card1
crw-rw---- 1 root video 226, 1 Jan 31 11:42 /dev/dri/card1
$ groups
pi adm dialout cdrom sudo audio video plugdev games users input netdev gpio i2c spi

That appears to be right to me, I’m in group video and group video has read/write access on card1. However if I just hit it with a hammer

$ sudo /usr/bin/kmscube -D /dev/dri/card1

then suddenly I have the beautiful rotating cube.

If I look slightly wider, I see that /dev/dri/renderD128 is in group render. Adding user pi to group render,

$ sudo usermod -a -G render pi

do the logout/login dance to pick up the new groups and suddenly the no-sudo kmscube works.

/usr/bin/kmscube -D /dev/dri/card1

And the reported information is more what you would expect

<snip>
OpenGL ES 2.x information:
  version: "OpenGL ES 3.1 Mesa 19.3.2"
  shading language version: "OpenGL ES GLSL ES 3.10"
  vendor: "Broadcom"
  renderer: "V3D 4.2"
<snip>

In future blogs I’ll take a look at the actual code.

Tagged with: , , ,
Posted in Uncategorized

A simple libCEC4 example for the Raspberry Pi

Many moons ago I wrote A simple libCEC example for the Raspberry Pi which worked fine until libcec4 came out in late October 2016. Yes, this post is long overdue.

As before, I managed to achieve great code reduction by tossing away almost all of cec-client’s features. I have managed to create a short (75 source lines of code count) example which is much easier to understand. The source code for this really short example can be found on github .

The basic idea of the program is

  1. Initialise libcec4
  2. Give libcec4 a callback that will handle the CEC keypresses
  3. Go to sleep until ctrl-c finishes the program

Any keys pressed on your remote control then wake up the callback function which I creatively called on_keypress.

The changes to the code from the prior version of libcec are fairly simple. First up, the on_keypress function now is passed a pointer to a cec_keypress message rather than a by-value copy of a message. The next change is that the CEC framework has changed the name of  the callback function from CBCecKeyPress to the simple keyPress.  In a similar vein, the FindAdapters function is now DetectAdapters.  And finally, the DetectAdapters function fills in an array of cec_adapter_descriptor rather than cec_adapter, which has the flow on effect to the Open call taking a strComName rather than simply comm.

One final note: this is simple tutorial code and is not production ready C++.  For example, in production code you would use RAII to encapsulate the LibCecInitialise/UnloadLibCec construction/destruction pair of functions.

 

 

 

Posted in Uncategorized

Using Travis-CI on python-compiletools

Background

If you buy into the concept that Unit Tests and Continuous Integration is a good thing then you will eventually bump into Travis-CI (https://travis-ci.org/).  The concept is that Travis-CI will take your github project (in my case https://github.com/Zomojo/compiletools) and run the unit tests.  All you need to do is add a small .travis.yml file that contains details like the language and commands to run your tests then the rest is supposed to be automagic.

Reality vs Marketing

With hindsight, I bet that for many projects Travis-CI works like a charm.  However I immediately failed. Now that I’ve finally cracked it, I can say that one of the reason I initially failed was due to the python requirements of my project and the other was a difference in hyphen handling by argparse.

My project was developed on Fedora 24 and Arch Linux, however the container that Travis-CI provides is Ubuntu based. As a specific example of dependency issues, I depend on configargparse. On F24 (Oct 2016), installing python-configargparse gives you version 0.10.  Ubuntu doesn’t provide configargparse in a package so you get  configargparse from PyPi (via pip install configargparse) which is now at version 0.11.  It turns out that there is a change in API between those two versions that broke my project.  That wasn’t the only difference between the two environments but it is a good example.

Experienced pythonistas will simply point out that I could have pinned the version of configargparse in my requirements.txt. And while that is true, it’s 1) not obvious to somebody with only hundreds of hours of python experience, and 2) wasn’t my only problem.

At this point a good bit of advice to my future self is “Use Travis-CI from the beginning”. If I had have done that my project would not have evolved into a place where it took almost two days to get running on Travis-CI.

Travis-CI tips

My (finally working) .travis.yml file is

language: python
python:
    - "2.7" 
    - "3.4"
    - "3.5"
addons:
    apt:
        sources:
            - ubuntu-toolchain-r-test
        packages:
            - g++-5
            - gcc-5

# command to install dependencies 
install:
    - pip install -r requirements.txt 
    - export CXX="g++-5" CC="gcc-5" 

# command to run tests
script:
    - nosetests

The language and versions of python are fairly obvious.  The addons section is only necessary because my project (python-compiletools) is about compiling C++ code so I need a C++ compiler available to run the tests.

The install section took a lot to learn.  There seems to be many ways of doing this section and one of my own hangups was that I didn’t want to write a requirements.txt that got out of sync with my setup.py.  I eventually learnt that if you have a requirements.txt that contains simply

-e .

then the requirements are extracted from setup.py.

The export line in the install section is a project specific requirement. That is, I need a compiler that can compile C++11 in order to complete my specific unittests.

Initial Giveup

It’s worth pointing out that at the time I gave up on using Travis-CI my project was (locally) passing unit tests and in active use on Fedora 22, 24, Arch Linux and Centos 7. So in one sense the code was “good”. Looking over my git logs I can see that I had 9 commits trying to make it work on Travis-CI before I threw in the towel. The process of reading Travis errors, making a commit that I think will work, waiting for Travis, reading more errors, was too slow and the big reason that I gave up.

Ubuntu

For unrelated reasons I set up an Ubuntu virtual machine on F24 using virt-manager. Having my hands on this environment then allowed me to fix a lot of the errors that Travis-CI was reporting. One of the stranger errors that occurred seems to be a difference in the algorithm that argparse uses to tell if an option starting with a hyphen is actually an option in its own right or if it was meant to be data for a prior option.  For example,

myexe --foo -bar

Is that meant to be a variable called foo is set to “-bar”  or is -bar another option?   A realworld example that I was facing is passing around options for g++

ct-cake --CXXFLAGS=-std=c++11

so the -std is not an option in its own right but a setting for my CXXFLAGS variable. That worked fine on F24 but failed on Ubuntu.

I eventually got around this particular problem by implementing a quote stripping phase in my program.  I now use a command line like

ct-cake --CXXFLAGS="-std=c++11"

Then my python code looks like

cap = configargparse.getArgumentParser()
args = cap.parse_args()
strip_quotes(args)

And the strip_quotes function removes the extraneous double quotes on any of the arguments. See around line 332 in https://github.com/Zomojo/compiletools/blob/master/ct/apptools.py if you are interested in how I implement the quote stripping.

As mentioned earlier there was also the difference in versions of python modules which I took care of by a try statement around the import.

At this point all my unit tests worked on Ubuntu so I thought I’d give Travis-CI another crack.  Unexpectedly, I still failed!

Docker

A bit more learning led me to the fact that the Travis-CI team make available a docker image that you can use to test on.  This answers my previous objection that I was taking too long in the guess solution, commit, push, wait, blowup, cycle.

I installed docker using

sudo dnf install docker-client docker --allowerasing 

and started docker with

sudo systemctl start docker

and proved that I’d got this far correctly via

sudo docker run hello-world

The docker image from Travis-CI came by running

docker run -it -u travis quay.io/travisci/travis-python /bin/bash

switched to the travis user (tip: user name is also the password)

su - travis

checked out my project

mkdir Zomojo
cd Zomojo/
git clone https://github.com/Zomojo/compiletools.git
cd compiletools/

I can’t remember why or from where I got this line of magic

travis compile

Travis-CI does its unittesting in a python virtualenv so we need to activate that

source ~/virtualenv/python2.7/bin/activate

Remember from my earlier .travis.yml I needed some specific environment variables so time to set those

export CXX="g++-5" CC="gcc-5"

And finally we can test and fix quickly

nosetests

Conclusion

Despite the rather large learning hurdles that needed to be vaulted, overall I’d recommend using Travis-CI.  In particular I recommend setting Travis-CI up very early on in your project so that you never reach the point of a wide divergence between tests that pass on your local environment and tests that pass on Travis-CI. Your code will be more robust by having the different environment running it and you are more likely to encounter and fix problems before your users run into them.

 

Tagged with: , , ,
Posted in Uncategorized

systemd –user, a first look

Version

The information in this post was tested on Fedora 22 which reports

$ systemctl --version 
systemd 219

Two Modes

systemd runs in both system mode and user mode.  In a corporate environment this is amazingly useful because it means you can set up your own user services without having to hunt down a system administrator to set up system services.

Hello World as a bash program

In my ~/bin directory I wrote a simple bash hello world program that once every 10 seconds writes hello world to a file in /tmp/.  Make sure you chmod u+x ~/bin/helloworld.

$ cat ~/bin/hello-world 
#!/bin/bash

rm /tmp/hw.txt
while true; do
 DATE=$(date +%Y%m%dT%H%M%S.%N)
 echo $DATE hello world >> /tmp/hw.txt
 sleep 10
done

Unit file

systemd services are specified in “unit files”. User unit files are placed in ~/.config/systemd/user/.  My first mistake toying about with systemd user mode was that I replaced the characters “user” with my username. Those characters really are meant to be “user”.

$ cat ~/.config/systemd/user/hw.service
[Unit]
Description=Hello World

[Service]
ExecStart=/home/geoff/bin/hello-world

[Install]
WantedBy=default.target

Note that the command that is run by ExecStart must be specified as a full path.  If you do something like

ExecStart=~/bin/hello-world

or try to be smart and use the systemd variables like ‘%h’

ExecStart=%h/bin/hello-world

then when you run

$ systemctl --user status hw

you will get errors like

Sep 16 14:48:27 ws002.zomojo.com systemd[25899]: [/home/geoff/.config/systemd/user/hw.service:6] Executable path is not absolute, ignoring: %h/bin/hello-world
Sep 16 14:48:27 ws002.zomojo.com systemd[25899]: hw.service lacks both ExecStart= and ExecStop= setting. Refusing.
Sep 16 15:39:36 ws002.zomojo.com systemd[25899]: [/home/geoff/.config/systemd/user/hw.service:5] Executable path is not absolute, ignoring: ~/bin/hello-world
Sep 16 15:39:36 ws002.zomojo.com systemd[25899]: hw.service lacks both ExecStart= and ExecStop= setting. Refusing.

Interestingly, when you fix those errors by putting the full path in the command line, then rerun systemctl –user status hw, you will still see all the old errors.  Don’t go bananas trying to figure out why systemd has failed to reload your unit file, all is actually well.  You can see all is well by 1) noting that the timestamps on the errors are exactly the same as they previously were and 2) try starting the service.

$ systemctl --user start hw

Watch the service run via

$ tail -f /tmp/hw.txt

And stop the service by

$ systemctl --user stop hw
Tagged with: ,
Posted in Uncategorized

A premade libcec deb

I’ve run across people who are interested in using libcec and would prefer to get started as soon as possible rather than going down the compile it up route.  For those people, you can download gzipped deb files from

http://www.owltra.com/libcec/index.html

Then gunzip the deb file and install

cd Downloads 
gunzip libcec_20150809-1_armhf.deb.gz 
sudo dpkg -i libcec_20150809-1_armhf.deb 
sudo ldconfig

A prerequisite is to have liblockdev1 installed.

sudo apt-get install liblockdev1
Tagged with: , ,
Posted in Uncategorized

Multiple strange errors involving SE Linux with Fedora 22

At work we are in the process of converting our developer workstations over to using Fedora 22.  I’ve had a few rather strange errors

  • SELinux is preventing sshd from read access on the file authorized_keys.
  • webcam not working
  • sound not working

One of the interesting features of our workstations is that we have the the home area under /data/work/<user> rather than /home/<user>. To make a long story short, I found an idea at http://www.geekheads.net/selinux-and-ssh/

Which lead me to

  • edit /etc/selinux/targeted/contexts/files/file_contexts.homedirs
    • copied the /home rules to new /data/work rules
  • ran restorecon -p -R /data/work
  • restarted, so that processes get the new contexts

The result being that webcam, sound, and ssh are now working.

Tagged with: ,
Posted in Uncategorized

CEC 3.0.0 on the Raspberry Pi

UPDATE 20180123

This article is now years out of date.  you should simply be able to install the cec-utils package and all should be well.

sudo apt-get update && sudo apt-get dist-upgrade
sudo apt-get install cec-utils

Background

In a previous blog I gave some instructions on how to compile up libcec on the RPi. Those no longer work as of CEC 3.0.0. In this post I’ll give you the new instructions that work as at 11 June 2015. To be specific, I have tested these instructions on an RPi2 running Raspbian. I have to say thankyou to nlrb for leaving excellent instructions on the RPi forums.

The first thing to do is update your RPi to the latest packages.

sudo apt-get update && sudo apt-get dist-upgrade
sudo rpi-update
sudo reboot

Next install the prerequisites for building libcec

sudo apt-get install git cmake g++-4.8 checkinstall liblockdev1-dev libudev-dev libxrandr-dev python-dev swig

Clone the Pulse-Eight libcec repository

Be careful to do the clone recursively otherwise you will be missing the “platform” submodule.

cd [workarea]
git clone --recursive https://github.com/Pulse-Eight/libcec.git

Patch src/libcec/cmake/CheckPlatformSupport.cmake

(remove the line with the -sign and add the lines with the +sign. Other lines are just included for context. Around about line 80) with

- check_library_exists(bcm_host vchi_initialise "" HAVE_RPI_API)
+ check_library_exists(bcm_host bcm_host_init "" HAVE_RPI_API)
 if (HAVE_RPI_API)
 set(LIB_INFO "${LIB_INFO}, 'RPi'")
+ include_directories(/opt/vc/include /opt/vc/include/interface/vcos/pthreads /opt/vc/include/interface/vmcs_host/linux)
 list(APPEND CMAKE_REQUIRED_LIBRARIES "vcos")

Create the cec platform library

cd [workarea]/libcec/src/platform
mkdir build
cd build

Note the dot dot at the end of the following line

cmake -DCMAKE_CXX_COMPILER=g++-4.8 ..
make
sudo checkinstall

Finally build libcec

cd [workarea]/libcec
mkdir build
cd build
cmake -DCMAKE_CXX_COMPILER=g++-4.8 -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=1 ..
make
sudo -s
export LIBRARY_PATH=/opt/vc/lib
checkinstall
Tagged with: , , ,
Posted in Uncategorized

A simple libCEC example for the Raspberry Pi

UPDATE:  See A simple libCEC4 example for the Raspberry Pi for more up to date information.

After getting the libcec test client (cec-client) to run my thoughts turned to incorporating CEC control into my own code.  The source code for cec-client is 1145 lines long, according to sloccount, which takes a bit of time to get your head around. By tossing away almost all of cec-client’s features I have managed to create a 75 sloc example which is much easier to understand. The source code for this really short example can be found on github .

The basic idea of the program is

  1. Initialise libcec
  2. Give libcec a callback that will handle the CEC keypresses
  3. Go to sleep until ctrl-c finishes the program

Any keys pressed on your remote control then wake up the callback function which I creatively called on_keypress.

Tagged with: , , ,
Posted in Uncategorized