Be Lazy, Make Computers to DO more :)

Sunday, March 22, 2015

Comparing Audio files with Sense

Hello Readers,

This week, I did an interesting (at least to me) experiment and study about how to compare the Audio files (Audio data).

Background

@Rajesh is my good friend at @Work and my morning walk buddy. He is a hard core Electronics guy , He gives lots of gyan about the Radio waves, AM signals etc etc .. Mostly I wont get it but he is patient enough to explain in plain words which can be understood by a computer science engineer (its me :)).

@work, we are working on Digital Radios, To simplify the scenario, imagine,

1. Radio station pick up the song - Which may be raw PCM data / Compressed as high quality MP3

2. Radio station transmits the song over the Air

3. Your home radio box receives it and reproduce the same Song - By now, it wont be in original file format

4. You will hear the song with acceptable quality

5. Great

@Work in the labs we simulate this situation with various equipments and test the song received is in acceptable quality.

@Rajesh, explain me the pain of this testing. Everytime we need to literally sit and listen the song to know that radio receiver is working fine and reproducing the song with good quality.

The main issue is that Songs / Real life record content is totally different than the audio test vectors (Ex: Sine wave tone). Some of the problems will only occurs to the Songs but in the same setup Sine wave tone works fine.

Obviously, @Rajesh told more about electronics and behaviour of the Radio Signals. I wont explain here (until I get to know it properly :) ).

We were talking about automation of audio listening tests. Due to the nature of the audio data, we can not compare them as a file comparison. Moreover in our situation, Source and Destination audio format is changing and Audio quality also changing. Opps lots of variables.

@Rajesh asked me to find some way to test it.

I was approching this problem from the computer science point of view.

My first thought was, we need to have some model which is similar to human ears. Irrespective of audio file format, quality changes, Humans can find out the similar songs.

Obviously, Google is the best friend while learning. I knew that, there are lots of music recognition applications available in mobile world. I have used Shazam, It could identify the songs properly.

I never gave a good thought about how Shazam kind of applications are working but now its needed for me.

I learnt that Shazam kind of music recognition apps are working using Acoustic fingerprint algorithms.

There are many Acoustic fingerprint algorithms are available. I chose the open source chromaprint algorithm developed by Lukáš Lalinský.

chromaprint converts the song / music as a musical notes and creates the fingerprints for various parts of the song. So the file format change and quality change is least affects this algorithm. Its a brilliant idea.

Read How chromaprint works, if you really interested in the algorithm.

chromaprint provides the pre built tool called fpcalc. fpcalc used to calculate the acoustic fingerprint. To install the fpcalc in Fedora linux,

[root@bakkisweety ~]# yum install libchromaprint-devel chromaprint-tools libchromaprint
Loaded plugins: langpacks, show-leaves, upgrade-helper
Package libchromaprint-devel-1.1-3.fc21.x86_64 already installed and latest version
Package chromaprint-tools-1.0-6.fc21.x86_64 already installed and latest version
Package libchromaprint-1.1-3.fc21.x86_64 already installed and latest version
Nothing to do

1
2
3

[root@bakkisweety Audacity]# fpcalc -version
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
fpcalc version 1.1.0

Note: Ignore the warning message from fpcalc

Now, I have a tool to find the acoustic fingerprints. I also need some test data to proceed. I have downloaded a public available song from youtube as MP3 and also recorded my own voice using Audacity.

If you like to work on MP3 files directly in Audacity, You need to install,

[root@bakkisweety ~]# yum install audacity-freeworld.x86_64
Loaded plugins: langpacks, show-leaves, upgrade-helper
Package audacity-freeworld-2.0.6-1.fc21.x86_64 already installed and latest version
Nothing to do

The normal Audacity package is not having support to import MP3 / MP4 audio.

I have created following files using Audacity,

1. hello_10.wav - My hello voice. Total length 10 sec

2. hello_10.mp3 - hello_10.wav is converted as a MP3 file

3. hello_10_noise1.wav - hello_10.wav, Added more noise in middle and small point in beginning.

4. hello_10_less_noise.wav - hello_10.wav, with less noise added.

Tools are installed, Test audio files are available, Lets starts the experiment,

[root@bakkisweety Audacity]# fpcalc hello_10.wav 
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10.wav
DURATION=10
FINGERPRINT=AQAAQWOiRRojodY0IZG-J0Eq7YLDo3cR86C4i5CbHXkvoXeRqVq4IDkfWL2EpmmK-0HYBUyebpA7HfmFOumERLryJEilXXB49C5iHg2XE1JWpkOuC3VuZKqmLUjOPbB6oQnTVDh9fDkyPd0gdzryC3XSItMVLgkS67hOVHUR82jIE1L2dMh1oc6NtJq2QDL3IFUvNKGPpym-Pch0b5AbIb-E3kWmKgoXIbE-XCeqNsXpIyxPAAGEA8wQQ4BQAgEEjBRAOMAQBEgQAYBBCBgpGBIMEQgQEQIBhIwDDAkAEA
[root@bakkisweety Audacity]# fpcalc hello_10.mp3 
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10.mp3
DURATION=10
FINGERPRINT=AQAAQWOiKNIYCXUyTUIifVeQSrvQhMddxDwo7iLkZkfeS-hdZKoWLkgsfoF_ognTFKePsMsJJk8HudORX6iTaUKm70mQUNrh8ETvIubRcLkIKWeO_LpQxzXSal2QyJwXWD2aME2F08eXI9PTDXKnI79QJy0yfVyCxPKJ60RVFzGPhssJKSvTIdeF50ZTTVuQyNyDVL3QhD6epvj2INO9QW6E_BJ6F5mqjEuCxPJxneib4j7CkCcAAYRTzBBDAAECAISAkQIYAxgRAgggADAIASMFQIIhCBAiQiCAkHEAIAEQQAA
[root@bakkisweety Audacity]# fpcalc hello_10_noise1.wav
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10_noise1.wav
DURATION=10
FINGERPRINT=AQAAQWOiKNIYCXXSCYl05UmQSrvg8Ohd5IezXYTEZjny6hJ6F5mqdUFy7oFVWWjyVPiDNsrRPJWKF0f4HuLH42GPTF-F5xeqoz3chCQ-HPmQ_PhOoaZwfqi1ywhzpobc6cgv9C4yVVG4JEis4zpR1UXMoyFPMHs6JNeF50ZTTSMSmbuCVL3QhMfjIt4O6hYhN0feS-hdZKqicBES6_gl1GmK-wjLAwGEA4YZYQAQxFCAEFMCCGIMAMBoiUQWAiCBBABGAoYEQ8QBhIBwCCFlCJBMIIAA
[root@bakkisweety Audacity]#

Ok. We have some acoustic fingerprints data but what do with this data. This acoustic fingerprints are encoded with specific format. We need a RAW acoustic fingerprints. Lets get the RAW acoustic fingerprints,

[root@bakkisweety Audacity]# fpcalc hello_10.wav -raw
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10.wav
DURATION=10
FINGERPRINT=1445291460,1450701268,1375130111,1369883486,1353107262,1357172526,1348782959,1092933959,1126558149,1445293508,1449616852,1475801597,1375130622,1353106270,1357238126,1365560174,1365571967,1126693191,1109748165,1445293508,1450668500,1375130111,1369883486,1353107262,1357172526,1348782959,1097136479,1126689221,1176857028,1449485780,1475867133,1375130622,1369883486,1357238062,1348782958,1365571950,1126693191,1109748165,1445293508,1449619924,1375130109,1370936190,1353107326,1357172526,1348782959,1097132383,1126689221,1176857028,1449485780,1475867125,1375130623,1369883486,1353172782,1357171502,1365563758,1126553927,1126525381,1445293508,1449616852,1408684541,1370936190,1353107326,1357238062,1348782958,1097132415

[root@bakkisweety Audacity]# fpcalc hello_10.mp3 -raw
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10.mp3
DURATION=10
FINGERPRINT=1445293508,1467478484,1375130111,1369883486,1353107246,1357172526,1348782959,1092933959,1126558149,1445293508,1449616852,1475801597,1370936190,1353106270,1357238062,1348782958,1097136511,1126693191,1109748165,1445293508,1450701268,1375130109,1370932062,1353107262,1357172526,1348782959,1092942175,1126689093,1176857028,1449616852,1475867125,1370936318,1369883486,1357238062,1348782958,1365571950,1126693191,1109748165,1445293508,1449619924,1442206205,1370936190,1353107326,1357172526,1348782959,1097136479,1126689221,1176857028,1449485764,1475867124,1375130623,1369883486,1353172782,1357171502,1365563758,1126553927,1126525381,1445293508,1449616852,1375097341,1370936190,1353107326,1357238126,1348782958,1097132383

[root@bakkisweety Audacity]# fpcalc hello_10_noise1.wav -raw
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10_noise1.wav
DURATION=10
FINGERPRINT=1445293508,1450668500,1375130111,1369883486,1353107262,1357172526,1348782895,1092933967,1126689221,1445293508,1449616852,1475867133,1375130622,1369867102,1357218606,1365541678,1348777278,-718818034,-718953202,-718951154,-735593185,-197747395,-734078147,-730969324,-781317164,-714208380,-680654444,-696314444,-696307276,-687918668,-692637259,-692767818,-744148490,-744196634,-773554842,1399122294,1126693191,1109748165,1445293508,1449616852,1375130109,1370936190,1353107326,1357172526,1348782959,1097132383,1126689223,1176857028,1449485764,1467478516,1375130623,1369883486,1353107246,1357171502,1348786543,1092999495,1126558149,1445293508,1449616852,1408684541,1375130494,1353106302,1357238126,1348782958,1365567871
 
[root@bakkisweety Audacity]# fpcalc hello_10_less_noise.wav -raw
fpcalc: /usr/lib64/nvidia-304xx/libOpenCL.so.1: no version information available (required by /lib64/libavutil.so.54)
FILE=hello_10_less_noise.wav
DURATION=10
FINGERPRINT=1445293508,1450668500,1375130111,1369883486,1353107262,1357172526,1348782895,1092933967,1126689221,1445293508,1449616852,1475867133,1375130622,1369883486,1357236014,1365561134,1365571966,1395128654,1445324878,1445325006,1445428447,1429624063,1362515327,1348880221,1357270604,1353075276,1369725916,1474587644,1449422308,1450468836,1475858924,1375130622,1369883486,1357367086,1357171502,1365571950,1126684999,1109748165,1445293508,1449616852,1375130109,1370936190,1353107326,1357172526,1348782959,1097132383,1126689223,1176857028,1449485764,1467478516,1375130623,1369883486,1353107246,1357171502,1348786543,1092999495,1126558149,1445293508,1449616852,1408684541,1375130494,1353106302,1357238126,1348782958,1365567871

Sweet, Now we have some numerical data. But still we need to make sense out this numerical data. For my situation, I need to see that two songs (one from the source which is radio station and other from radio receiver) are matching with in the threshold. For sure, The two songs wont be same.

Lets consider the numerical (signed 32bit integers) fingerpoint data as a data sets. So we have two sets in our hand. I choose the R Square assess of goodness of fit method.

In my situation, Since the source song and destination song content is same, I should get a positive correlation and in ideal lab conditions, R square should be 1.0 but so many things will happen during the radio transmission. So we can't get 1.0 but the it should be grater than 0.0. If I get 0.0 as a R square, something really wrong happen during the trasnmission.

Dont ask me why I have chose goodness of fit statistics method instead of something in electronics domain like FFT and its derivatives... I yet to test it with real data. I chose it based on pure instinct.

I wrote a perl script to do the whole process of running fpcalc to get the fingerprints, Calculate the R square and report the goodness of fit. Here is the script,

#!/usr/bin/env perl
#Script to calculate the similarity of the two audio files using chromaprint fingerprint algorithem 
#License: LGPL2.1+
#Author: Bakkiaraj M [http://npointsolutions.blogspot.in/]
#Usage: audio_chromaprint_diff.pl audio1_file audio2_file
#Note: Install the fpcalc tool version 1.1 before you run this script 

#Refer URLs
#Refer: http://acoustid.org/chromaprint
#Refer: http://en.wikipedia.org/wiki/Coefficient_of_determination

use strict;
use warnings;
use Statistics::LineFit;
use Data::Dumper;
use Capture::Tiny ':all';

#Globals
my $fp1ArrRef;
my $fp2ArrRef;
my $audioLen = 9999; #Len in secs for fingerprint calculation. Just give rough high number. Later enhance based on songs.

my $fpcalctool = '/usr/bin/fpcalc -raw -length '.$audioLen.' ';

#Get the file names from the command line 
my $fn1 = $ARGV[0];
my $fn2 = $ARGV[1];

my $fn1FPs = 0;
my $fn2FPs = 0;
my $fpDiff = 0;

if (!defined ($fn1) or !defined ($fn2))
{
 print "\n Usage: perl $0 audio_file1 audio_file2";
 exit (-1);
}

if (! -s $fn1 or ! -s $fn2)
{
 print "\n ERROR: $fn1 or $fn2 is not a proper file.";
 exit (-1);
}


#Function
sub calcFingerPrint
{
 my $fileName = shift @_;
 
 #print "\n EXEC: $fpcalctool \"$fileName\"";
 
 my ($fpdata, $stderr, $exit) = capture {
  
        system( $fpcalctool.' "'.$fileName.'"' );
     };
    
    unless ($exit == 0)
    {
     print "\n ERROR: While running fpcalc tool";
     print "\n CMD: ",$fpcalctool.' "'.$fileName.'"';
     print "\n STDERR: ", $stderr;
     exit ($exit);
    }
    
 if ($fpdata =~m/FINGERPRINT=(.*)/g)
 {
  my @fpDataArray = ();
     @fpDataArray = split (/,/,$1);
     return \@fpDataArray;
 }
 else
 {
  return [];
 }
}

$fp1ArrRef = calcFingerPrint($fn1);
$fn1FPs = scalar @$fp1ArrRef;
$fp2ArrRef = calcFingerPrint($fn2);
$fn2FPs = scalar @$fp2ArrRef;

#$,=" ";
print "\n File1: $fn1 Tot FPs ",$fn1FPs;
#print "\n ", @$fp1ArrRef;
print "\n File2: $fn2 Tot FPs ", $fn2FPs;
#print "\n ", @$fp1ArrRef;

if ($fn1FPs != $fn2FPs)
{
 #Equalise th array items , so it will be equal.
 if ($fn1FPs > $fn2FPs)
 {
  splice ($fp1ArrRef,$fn2FPs);
 }
 else
 {
     splice ($fp2ArrRef,$fn1FPs);
 }
    print "\ Note: File1 & File2 have different FingerPrints, Lowest number of Fingerprints will be used in R2"; 
}



my $lfit = Statistics::LineFit->new();
$lfit->setData($fp1ArrRef, $fp2ArrRef);

printf "\n\n Goodnees of fit R2 for File1 & File2 = %.8f \n", $lfit->rSquared();

exit (0);

This script takes the two audio files and calculate the goodness of fit based on the acoustic fingerprints data. Lets run it,

[root@bakkisweety Audacity]# /mnt/WinD/Eclipse_WorkSpace/PerlExamples/audio_chromaprint_diff.pl hello_10.wav hello_10.mp3 

 File1: hello_10.wav Tot FPs 65
 File2: hello_10.mp3 Tot FPs 65

 Goodnees of fit R2 for File1 & File2 = 0.92182237 
[root@bakkisweety Audacity]# /mnt/WinD/Eclipse_WorkSpace/PerlExamples/audio_chromaprint_diff.pl hello_10.wav hello_10_less_noise.wav 

 File1: hello_10.wav Tot FPs 65
 File2: hello_10_less_noise.wav Tot FPs 65

 Goodnees of fit R2 for File1 & File2 = 0.50358946 
[root@bakkisweety Audacity]# /mnt/WinD/Eclipse_WorkSpace/PerlExamples/audio_chromaprint_diff.pl hello_10.wav hello_10_noise1.wav 

 File1: hello_10.wav Tot FPs 65
 File2: hello_10_noise1.wav Tot FPs 65

 Goodnees of fit R2 for File1 & File2 = 0.02005060

The output clearly shows that my method of comparing the audio files is not greatly affected by the audio file formats (MP3 / WAV).

When I calculate the goodness of fit for hello_10.wav & hello_10.mp3, I get 92%, which shows that these are almost same songs.

When more noise added the goodness of fit goes low.

I have also tested with real songs / music files. The outcome is convincing.

Can we automate the audio listening test with this approach?

Yes. we need to already set the goodness of fit tolerance level before we start the audio listening test.

Goodness of fit tolerance level is based on the source song, compressing techniques, transfer medium (AM / FM) etc ... Which I still need to workout.

I hope, with this method, we can reduce the human audio listening times during the testing.

Nevertheless, Nothing can replace human ears when it comes to music / audio testing. The above kind of methods only assists and provides the likely hood of problematic test vectors.

I will update once I work real captured data in the lab.

If this method works well, I am planning to create a perl module to wrap the libchromaprint functionality. so we can call the fpcalc as a perl function instead of subprocess.

Bye Bye ...

Sunday, March 8, 2015

Notify to your user in Linux - GNotification

Hello Readers,

In the last post, I was referring to GNotification. Here is the dedicated post of my experiments with GNotification APIs with Perl.

Notification - A pop up message to show the quick details about what is going in the application. Usually in both linux and windows, The notifications are shown in lower right corner (system tray) of the desktop. In Linux GNOME desktop, its usually upper right corner.

Linux KDE desktop notification looks something like this,

Note: The notification appearance may vary based on the theme that you set in your desktop.

In Windows OS, Desktop notification looks something like this,

I was looking for a good way to notify from my perl application to user in Linux. In windows os, There is only one way to do. Call windows APIs yourself. But we are in free world, Linux, We have more than one way to do.

1. GNotification - Notification APIs provided by GNOME and tightly coupled for GTK applications but not hard to use in any applications even command line application. The main advantage of the GNotification APIs is its a persistence notification. Even after the system crash / reboot user will see the notification until they respond to it.

GNotification can also create notifications like questions that user can respond.

2. Libnotify - Notification APIs provided by GNOME. Very simple to use and its kind of standalone api set not tightened up with GTK. Simple and less powerfull.

We will use the GNotification API with Perl application with the help of our best foreign library interaction perl module - FFI::Platypus.

GNotification Way of Working is,

1. Create a new GTK application and gets its application ID

2. Register the application

3. Create Notifications

4. Set the Notification parameters

5. Show the Notifications to the user

6. Clean up everything.

Note: If you write a native GTK application, Step #1, #2 is not necessary because its implicit.

Ok. Enough talking, Here is the code.

#!perl
#Description: Create GNOME Desktop notifications using GNOME libgio library and FFI::Platypus
#Refer: https://developer.gnome.org/gio/2.42/gio-GNotification.html
#Author: Bakkiaraj Murugesan
use strict;
use warnings;
use FFI::Platypus;
use FFI::Platypus::Declare;
use FFI::CheckLib;
use Data::Dumper;

$|++;
my $ffiObj = "";

print "\n Perl FFI::Platypus Gnome Notification Example";
#Find the lib
my $libPath =  find_lib( lib => '*', verify => sub { $_[0] =~ /^gio-2\.[0-9]+/ },libpath=>'/usr/lib64' );
my $gobjectLibPath = find_lib(lib=>'*',verify => sub { $_[0] =~ /^gobject-2\.[0-9]+/ },libpath=>'/usr/lib64');
my $glibLibPath = find_lib(lib=>'*',verify =>sub { $_[0] =~m/^glib-2\.\d+/;},libpath=>'/usr/lib64');

#Find the library Paths
print "\n Found libgio-2.0 in :", $libPath;
print "\n Found libgobject-2.0 in :", $gobjectLibPath;
print "\n Found libglib-2.0 in :", $glibLibPath;

#Create FFI::Platypus object
$ffiObj = FFI::Platypus->new();
$ffiObj->lang('C'); #FFI supports more than C language
$ffiObj->lib($libPath);

my $ffiObj1 = FFI::Platypus->new();
$ffiObj1->lang('C'); #FFI supports more than C language
$ffiObj1->lib($gobjectLibPath);

my $ffiObj2 = FFI::Platypus->new();
$ffiObj2->lang('C'); #FFI supports more than C language
$ffiObj2->lib($glibLibPath);

#Create gapplication
#g_application_new (const gchar *application_id,GApplicationFlags flags);
$ffiObj->attach('g_application_new',['string','int'],'opaque');

#Check application ID is valid 
$ffiObj->attach('g_application_id_is_valid',['string'],'int');
#Set GApplication Name
my $appName = 'perl.ffi.platypus.gnome.notify.example';
print "\n App Name $appName is valid? ",g_application_id_is_valid($appName); 
#G_APPLICATION_FLAGS_NONE is 0
my $GApplicationPrt = g_application_new ($appName,0);

#Register the application
#gboolean g_application_register (GApplication *application, GCancellable *cancellable, GError **error);
$ffiObj->attach('g_application_register',['opaque','opaque','opaque'],'int');
#GCancellable *g_cancellable_new (void);
#$ffiObj->attach('g_cancellable_new',['void'],'void');
#my $GCancellablePtr = g_cancellable_new();
print "\n Application Registration ID:", g_application_register($GApplicationPrt,undef,undef);

print "\n Send Welcome notification";
#Create GNotification                        
#GNotification *g_notification_new (const gchar *title);
$ffiObj->attach('g_notification_new',['string'],'opaque');
my $GNotificationPtr = g_notification_new('Welcome');

#void g_notification_set_body (GNotification *notification, const gchar *body);
$ffiObj->attach('g_notification_set_body',['opaque','string'],'void');

g_notification_set_body ($GNotificationPtr, "Welcome to GNotifications using perl + FFI::Platypus...");

#void g_application_send_notification (GApplication *application, const gchar *id, GNotification *notification);
$ffiObj->attach('g_application_send_notification',['opaque','string','opaque'],'void');
g_application_send_notification($GApplicationPrt,'perl.ffi.platypus.gnome.notify.welcome',$GNotificationPtr);

print "\n Send Urgent notification";
#void g_notification_set_urgent (GNotification *notification, gboolean urgent);
$ffiObj->attach('g_notification_set_priority',['opaque','int'],'void');
$GNotificationPtr = g_notification_new('Urgent');
g_notification_set_body ($GNotificationPtr, "This is Urgent Notification. Have a look at me!");
g_notification_set_priority($GNotificationPtr,2);
g_application_send_notification($GApplicationPrt,'perl.ffi.platypus.gnome.notify.urgent',$GNotificationPtr);

print "\n Notification with Buttons";
$GNotificationPtr = g_notification_new('Wanna Coffee?');
g_notification_set_body ($GNotificationPtr, "Would you like to drink Coffee?");

                 
#void g_notification_add_button (GNotification *notification,const gchar *label,const gchar *detailed_action);
$ffiObj->attach('g_notification_add_button',['opaque','string','string'],'void');
g_notification_add_button ($GNotificationPtr, "Ok. Sure", "app.ok");


g_application_send_notification($GApplicationPrt,'perl.ffi.platypus.gnome.notify.coffee',$GNotificationPtr);
                        
print "\n Clean Up everything";
#Do clean up
#void g_object_unref (gpointer object);
$ffiObj1->attach('g_object_unref',['opaque'],'void');
g_object_unref ($GNotificationPtr);
g_object_unref ($GApplicationPrt);

print "\n Ta Ta from PID:", $$;
print "\n Note: GNotification is persitant, Even after this application quit, System reboots, Notification will be there until user acknowledge it.";
#All is Well Bye
exit 0;

Here is the stdout looks like,

 Perl FFI::Platypus Gnome Notification Example
 Found libgio-2.0 in :/usr/lib64/libgio-2.0.so.0.4200.2
 Found libgobject-2.0 in :/usr/lib64/libgobject-2.0.so.0.4200.2
 Found libglib-2.0 in :/usr/lib64/libglib-2.0.so.0.4200.2
 App Name perl.ffi.platypus.gnome.notify.example is valid? 1
 Application Registration ID:1
 Send Welcome notification
 Send Urgent notification
 Notification with Buttons
 Clean Up everything
 Ta Ta from PID:3768
 Note: GNotification is persitant, Even after this application quit, System reboots, Notification will be there until user acknowledge it.

Here is the screen shot of the notifications generated by the above code,

Conclusion. Notify your user with as much as meaningful information. Do not make them to think that your application is not responding and doing something crap !!.

Bye.

Sunday, March 1, 2015

Running Win32 specific perl code in Linux

Hello Readers,

My wife and my son decided to give me a short "me" time. They are out of town for this whole month. I have planned to watch lots of movies and do some more experiments with perl + FFI.

As usual, I will update the experiment results here. Stay tuned.

WINE - Wine is an open source project. It aims to provide a Windows compatibility layer in Linux. So we can run the windows applications with in linux without going through Virtualization or any other tough route.

I must admit that I never really had any success of running useful windows application in my Fedora Linux via WINE. It appears that WINE can properly emulate core APIs but not fully the .NET kind of APIs, Also many guys out there using WINE for running Windows games in Linux Box. Games usually use the raw and core APIs of the OS.

I have a server grade computer with dual boot. One OS is obviously Fedora Linux 21 and other is Windows 7. I hate to login to Windows and my default login is set to Fedora Linux. Most of the time, I will finish my work in Linux OS itself and @Work I force to use Windows 7 laptop anyway. So I will defer my windows releated tasks to @Work laptop.

This week end, I was doing some experiments with perl and I had a situation to Win32 module. I can not do it Linux anyway because Win32 module, As name suggest needs windows OS to run. Win32 module exports a set of Windows OS APIs in to perl world.

In my other dual boot windows os, I have Active State perl v5.21 with all the perl development modules installed. But I don't like to shutdown and boot into Windows OS because of many days I did not login, It will complain about missing patches and older Virus Scanner. (Virus scanner!! another piece of crap, It will slow down the computer like anything and in Linux, We ask what is Virus? :))

Hmm... I have one more choice. In the Linux OS, I have KVM based windows XP virtual machine. I thought to start the Windows XP virtual machine and continue my perl experiments. After I logged into Windows XP virtual machine, I realize that, I do not have my perl development environment setup in it.

Like some people can not sleep in any beds and need their own bed and pillow, I need my development setup else my brain won't let me work. It keep points advantage of my development environment.

By the way, My perl development environment is,

Windows OS: Active Perl 5.20 +Eclipse IDE + EPIC Plugin + PPM for package management + gcc compiler + dmake

Linux OS: System default perl+ Eclipse IDE + EPIC plugin + CPAN for package management + gcc compiler + gmake

So, Windows XP Virtual Machine also not useful for now. I am too lazy to re setup my development environment in the Windows XP Virtual Machine.

Since I have the Dual Boot, I have also mounted my Windows drives (C:, E:) into my Linux via following entries in /etc/fstab file.

1
2

/dev/sda3 /mnt/WinD ntfs-3g rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other 0 0
/dev/sda2 /mnt/WinC ntfs-3g rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other 0 0

This allows me to access all Windows OS installed applications in /mnt/WinC. I finally thought to give a try with WINE.

Mission: Run Active Perl 5.20 which is installed in Windows OS in Linux using WINE.

First we need a WINE installed, I have installed long long back (May be since 2003, I have WINE in the Linux system without much usage because I am not a good computer games fan).

1 2	wine --version wine-1.7.36 (Staging)

Next need to access the Active Perl installation in Windows OS. Using /etc/fstab file, Windows hard disk partition (NTFS) also mounted in Linux under /mnt folder.

Finally run the Active State perl,

$ cd /mnt/WinC/Perl/bin/
$ wine perl -v
fixme:winediag:start_process Wine Staging is a testing version containing experimental patches.
fixme:winediag:start_process Please report bugs at http://bugs.wine-staging.com (instead of winehq.org).
fixme:ole:CoInitializeSecurity ((nil),-1,(nil),(nil),1,2,(nil),0,(nil)) - stub!
err:ole:CoGetClassObject class {0000034b-0000-0000-c000-000000000046} not registered
err:ole:CoGetClassObject no class object {0000034b-0000-0000-c000-000000000046} could be created for context 0x1
fixme:ole:RemUnknown_QueryInterface No interface for iid {00000019-0000-0000-c000-000000000046}

This is perl 5, version 20, subversion 1 (v5.20.1) built for MSWin32-x86-multi-thread-64int
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2014, Larry Wall

Binary build 2000 [298557] provided by ActiveState http://www.ActiveState.com
Built Oct 15 2014 22:10:49

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
$ fixme:crypt:ProvStore_release Unimplemented flags 1
fixme:crypt:MemStore_release Unimplemented flags 1
fixme:msvcrt:__clean_type_info_names_internal (0x36e3f8) stub
fixme:msvcrt:__clean_type_info_names_internal (0x10046a20) stub

Hurrah, perl runs smoothly. But WINE spits more debug messages, Its quite distractive. Put off the WINE debug messages with WINEDEBUG environment variable.

$ export WINEDEBUG=-all
$ wine perl -v

This is perl 5, version 20, subversion 1 (v5.20.1) built for MSWin32-x86-multi-thread-64int
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2014, Larry Wall

Binary build 2000 [298557] provided by ActiveState http://www.ActiveState.com
Built Oct 15 2014 22:10:49

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

Sweet :) Now, Perl runs as if I am running in Windows. But my main motive is to use the Win32 module, I still need to see WINE + Windows Perl can load the Win32 module.

Before that, I need to setup few environment variables for Windows Perl. Though WINE passes all the environment variables to running application due to OS difference few environment variables like PATH can not passed from Linux to Windows process running in WINE.

Special WINE environement variables needs to be exported via WINE Registry editor. It can be invoked via 'regedit' command OR if you are using KDE / GNOME desktop, You can get it in the start menu.

You can see that I have set the PATH variables with perl specific values. Z: is created by WINE, It is the Linux "/" folder.

Lets run Win32 code,

1
2
3

$ wine perl -e 'use Win32; print Win32::GetOSDisplayName()'

Windows XP Professional Service Pack 3

Awesome. Looks like WINE is emulating Windows XP.

This made me think, why perl can run peroperly under WINE?. Other commercial applications like Adobe Photoshop fails to run under WINE.

This is because Perl is super portable and it is mostly using libC (C runtime) core APIs (and modules will use other APIs, Depends on the Module) and perl as a language do not have may dynamic libs to link with. Basically, Perl is a low dependency language unlike MS Languages (.NETs).

No wonder perl is running in Over 100+ OS !!!!

After a decade, I made use of WINE and I will continue to use with Windows specific perl expriments.

Just now, Got a call from wife and son, Accusing me that, Even they are not in home, I finish blog in late sunday night and its obvious that they are not responsible for me to work in late nights!! True.

Hey, I am also doing experiments with GNotification APIs. Let me update in upcoming blog posts.

Bye Bye folks.

Sunday, February 1, 2015

Forgin Function Interface with Perl

Dear Readers,

Very Happy New year 2015. Opps!! I didn't realize that already a month over after the new year is started. Its Feb 2015 now. Hmmm.. Time pass by so fast.

After the Christmas and new year vacation, @work + @home keeps me busy and did not get enough time to do experiments.

@Work, we are working on a LPC micro controller based board to control another electronics board which has a digital radio IC (DAB | DRM | HD) on it. LPC micro controller board has a USB connectivity and it will be connected to Computer to take batch commands.

Batch commands the will get processed further inside the LPC micro controller board and the send to connected digital radio IC. This complex setup was made because we cant not afford PC to USB latency So we have a LPC board in middle. (Hey, I am not an embeeded enginner, So no more questions :) ).

We have also decided to use WinUSB based minimal driver in PC side to communicate with LPC micro controller. So there will be a .dll (Say liblpccommunicate.dll) which is wrapper over WinUSB provides communication functions.

In C world, its awesome. Just load the dll (and corresponding header file) and start communicating with LPC board (atleast in windows its easy).

How do we communicate with LPC board from scripting languages like Perl / Python?

My experiment is started here.

Earlier projects, we use to expose a virtual COM / Serial port from the USB drivers so all the scripting languages can just use the well defined COM / Serial interface to communicate from computer to device.

I have used Win32::SerialPort (for windows) perl module to communicate to devices (mostly electronic boards which has a specific SoC to test).

There is also a Device::SerialPort for Linux for the same purpose.

Now, I can not use the above super reliable modules from my perl scripts because of lack of COM port interface.

I was experimenting with multiple options to import the function from DLL / Shared Lib to perl's space and call it.

Some time back I was experimenting the very limited and windows only modules called Win32::API. The main limitation is that we can not call a function that returns a structure.

In my case, almost all the USB function would return a structure. So Win32::API is ruled out.

Then I came across a open source project (actively maintained since 1998!!) called libffi. libffi provides easy mechanism to call the functions exported in the DLL / Shared Lib at run time.

Awesome, I was looking for this !!. Next, I was looking options to use the libffi in perl. I dont want to re-invent the wheel again. So I search in metacpan , This is a beauty of perl. If you think you have a problem, some one would have already solved in CPAN so you can concentrate on your application work.

I found a module called FFI::Raw , I did few experiments and its seems to be working for me.

Then from the perl news letter, I got to know the new perl + FFI work by @Graham Ollis. He just released his new FFI module FFI::Platypus.

FFI::Platypus allows to import the function from the DLL / Shard Lib to the perl's namespace. This provides increased speed while calling the function. Also, I can easily pass and get the C struct.

HERE you can find some more advantages of FFI::Platypus.

Thanks to @Graham Ollis for his awesome work. This module is in still early stage and @Graham Ollis is keep adding new functionalities (Like record to pass pointer to struct in a polished way). I hope soon this module will be a defacto standard way to use FFI from perl.

I have started contributing to FFI::Platypus by adding example scripts to its github repository (and also asking more questions to @Graham Ollis :)).

Ok. Enough talking I guess, Let me show few examples of using the
FFI::Platypus.

Ohh wait before you see an example, How do we use C Structure in perl? Again, in perl there is a more than one way to do.

1. Using perl's in build pack & unpack - you get raw speed and power with pain of fiddling with pack templates.

2. use Convert::Binary::C - Simple to be used. Just directly use your C code.

Script 1 - Get system up time using linux GNOME's libgtop library

#!perl
#Description: Get linux system uptime using GNOME libgtop library and FFI::Platypus
#Refer: https://developer.gnome.org/libgtop/stable/libgtop-Uptime.html
#Author: Bakkiaraj M
use strict;
use warnings;
use FFI::Platypus;
use FFI::CheckLib;
use Convert::Binary::C;
use Time::Seconds;
use Data::Dumper;

#Globals
my $ffiObj = "";
my $libPath = "";
my $cStructObj = "";
my $glibtopUptimeSize = 0;
my $glibtopUptimeStruct = undef;
my $packedglibtopUptimeStruct = undef;
my $timeObj = "";

#Find the lib
$libPath = find_lib(lib=>'gtop-2.0',libpath=>'/usr/lib64');


print "\n Found libgtop in :", $libPath;

#Create FFI::Platypus object
$ffiObj = FFI::Platypus->new();
$ffiObj->lib($libPath);

#Create Convert::Binary::C object to import the structures 
$cStructObj = Convert::Binary::C->new();
$cStructObj->configure( 'Alignment' => 0 );

#import glibtop_uptime struct using Convert::Binary::C
#Note: guint64 is unsigned long as per 
#http://www.freedesktop.org/software/gstreamer-sdk/data/docs/latest/glib/glib-Basic-Types.html#guint64

$cStructObj->parse(<<ENDC);
struct glibtop_uptime {
    unsigned long flags;
    double uptime;      /* GLIBTOP_UPTIME_UPTIME */
    double idletime; /* GLIBTOP_UPTIME_IDLETIME */
    unsigned long boot_time;
};
ENDC

$glibtopUptimeStruct = {};
$packedglibtopUptimeStruct = $cStructObj->pack('glibtop_uptime',$glibtopUptimeStruct);

#Get size of the glibtop_uptime
$glibtopUptimeSize = $cStructObj->sizeof('glibtop_uptime');

#typecast the glibtop_uptime as a FFI::Platypus record
$ffiObj->type("record($glibtopUptimeSize)"=>'glibtop_uptime');
#import glibtop_get_uptime function from libgtop to perl
$ffiObj->attach('glibtop_get_uptime',['glibtop_uptime'],'void');

#Call glibtop_get_uptime
glibtop_get_uptime ($packedglibtopUptimeStruct);

#unpack the structure
$glibtopUptimeStruct = $cStructObj->unpack('glibtop_uptime',$packedglibtopUptimeStruct);
#print "\n", Dumper($glibtopUptimeStruct);
print "\n System is upfor: ", $glibtopUptimeStruct->{'uptime'}," Sec";

$timeObj = Time::Seconds->new($glibtopUptimeStruct->{'uptime'});
print "\n System is upfor: ",$timeObj->pretty;

#using uptime command
print "\n\n System is upfor:";
system('uptime -p');

exit (0);

Script 2 -Get System time from windows OS using GetLocalTime API

#!perl
# Author : Bakkiaraj M
# Script: Get System time from windows OS using GetLocalTime API.
use strict;
use warnings;
use FFI::CheckLib;
use FFI::Platypus;
use Convert::Binary::C;

#Get the system time using Kernel32.dll

#find the Kernel32.dll 
my $libPath = find_lib(lib=>'Kernel32');
#Create FFI Object
my $ffiObj = FFI::Platypus->new();
$ffiObj->lib($libPath);

#Import the GetLocalTime function
$ffiObj->attach('GetLocalTime',['record(16)'],'void');

#Define SYSTEMTIME Struct as per https://msdn.microsoft.com/en-us/library/windows/desktop/ms724950(v=vs.85).aspx
#As per, C:\MinGW\include\windef.h, WORD id unsigned short
my $c = Convert::Binary::C->new->parse(<<ENDC);
  
struct SYSTEMTIME {
  unsigned short wYear;
  unsigned short wMonth;
  unsigned short wDayOfWeek;
  unsigned short wDay;
  unsigned short wHour;
  unsigned short wMinute;
  unsigned short wSecond;
  unsigned short wMilliseconds;
  };
  
ENDC


my $dateStruct = {
  wYear=>0,
  wMonth=>0,
  wDayOfWeek=>0,
  wDay=>0,
  wHour=>0,
  wMinute=>0,
  wSecond=>0,
  wMilliseconds=>0,
};

my $packed = $c->pack('SYSTEMTIME', $dateStruct);

#Call the function by passing the structure reference
GetLocalTime($packed);

if (defined ($packed))
{
 #Unpack the structure 
 my $sysDate = $c->unpack('SYSTEMTIME', $packed);
 print "\n WINDOWS SYSTEM TIME: ",$$sysDate{'wHour'},':',$$sysDate{'wMinute'},':',$$sysDate{'wSecond'},'.',$$sysDate{'wMilliseconds'},' ',$$sysDate{'wDay'},'/',$$sysDate{'wMonth'},'/',$$sysDate{'wYear'}, "\n";
}
else
{
 print "\n Something is wrong\n";
}

exit 0;

For more examples go to HERE.

I hope, with libffi we can open up the scripting interface from our WinUSB based driver.

I will share the details in future..

Sunday, December 21, 2014

The concept of Software "Build Announcement"

Preface:

I originally wrote this article in http://www.cmcrossroads.com/ during June 2009. But now, I found that this article is missing from the http://www.cmcrossroads.com/ and I try to trace it.

I also found that my original article was referred by computer science department of Rochester Institute of Technology, NY.

Since the concept discussed in the article is still relevant, I thought to restore my article to my own blog (this one) with minor updates. I used Way Back Machine, to trace the original article source and I found it HERE.

I will also try to notify R.I.T CS department so that they may update the web link.

Lets continue to the article ...

The concept of Software "Build Announcement"

This is a new era in Software Build and Release management. Continuous integration brings more frequent software builds. More frequent software builds means more frequent updates for testers, developers, and other stakeholders. This brings out new use cases / user stories:

As a Test/Validation engineer, I want to know, is the "Product Build output" valid to take forward for testing.

As a Developer, I want to know, did the product build go through without any compilation issues? Was the Integration done correctly?

As a stakeholder (Product owner, Program manager), I want to know, what is the status of product build?

Many build management tools (e.g. Jenkins) provide the methods for notifying task completion via e-mails and they also provide a dashboard that shows task status. But "All green" status does not always mean that the product build is "good" and can be promoted to the testing phase.

This article is about creating effective Software "Build Announcements".

Let's understand the user stories / use cases,

What does SW validation / test engineer want to know?

Typically, a Software testing environment is a controlled (separate) environment from the production Software Build environment. So when the product needs to be tested, Tester / validation engineer will promote the product to the testing environment (which may be in a different subnet, with different firewall settings etc..) and the Software product needs to be installed in right environment with all its dependencies (older versions may need to uninstalled before installing latest build) and then, finally, it is ready for testing or validation.

Clearly the above steps involve efforts (which may raise questions), What if installer itself is broken? What if the one component failed to compile? What if the application is not starting at all?, Test engineer will find out anyway but then it is too late. The testing environment needs to be rolled back (Uninstall the latest build and install the previous build) in order to continue with the testing.

What does the Software developer want to know?

Software developers want to know that, the software build went through without compilation issues and components are integrated properly. Just parsing (e.g. looking at error patterns) compiler logs is not good enough because software products usually depend on many run time binaries and integration issues will be found only while using (testing) the software product.

Software developers depend upon the testing team to know the major integration issues or they themselves need to do the minimal testing (Sanity testing) to ensure that the software build is properly integrated.

What do all the stakeholders want to know?

Stakeholders (e.g. Product owners, program managers etc...) want to know the status of the product build in a rather "Boolean" way (Pass or Fail) because the build management tool's dashboard will not show the clear status. For example, compilation task may show green but the run time component integration may fail.

What about automated task completion notification?

Build management tools supports automated task notification mails. It just notifies us that a particular build / task has been completed. It is also for a particular task not for a complete product build.

The level of continuous integration differs from product to product. Less "weighty" products may build the whole product (including packaging) every time the source code changes. Heavy weight products (which can takes 6 to 14 hrs to build the complete product package) still depends on nightly builds for full build and it might build a few components for every source code change.

Either way, once the complete build is over, it is available for testing and then someone/something needs to ensure the quality of build.

So how do we satisfy all the above requirements? How do we make sure that everyone (Test Engineer, Developer and Stakeholders) gets the required information.

Software Build Announcement

Here we need the process of "Software Build Announcement". This is the mail notification that contains the "Boolean" result of build status (Pass/Fail).

How do we assess that the build has passed/failed? - It should be based on the "Quality Gate", You can name it as a BAT (Build Acceptance Test) or Sanity Test quality gate.

I prefer the "Sanity test" because it verifies the overall sanity of product which involves some part of functionality testing.

Sanity Testing

"Sanity Test" quality gate should involve the mandatory test cases which will help to ensure the build's status.

For example, A software product can have the following test cases to ensure the product is ready to take forward to testing,

1.   Compiler logs - Should not have compiler issues
2.   Packaging logs - Should not have packaging issues
3.   Product installation - Product should install with out issues
4.   Product invocation - Product should come up without crash
5.   Product de installation - Product should un install with out issue

Hey, Wait! Many test cases also involve testing functions, Should I test the product build before announce the build?

Yes!, Here comes the Joy of Automation. Automate your Sanity test cases and Build announcement.

Sanity Test Automation

In Nokia, (I was working in Nokia SW division in 2009, Now working in NXP Semiconductors, Automotive SW Division) We have automated both sanity testing and Build announcements. Sanity test frame work architecture looks like this,

As the architecture diagram explains, Once the product build is completed, the Build server invokes the Sanity test cases via the Sanity test framework and the build will be tested for sanity.

If all the mandatory test cases are passed, the build will be shared in a production build share area and the build will be announced as a "successful" build via Email - else the build will be shared in a temp build share and it will be announced as "failure" build via Email.

A minimal sanity testing helps here to announce the builds.

Here is our success build announcement format,

Here is our success build announcement format,

Sub: Build Announcement: Sprint_<Sprint No>_<Product_Name> -- build <Build Number>

<Product> Sanity Test Report
 -----------------------------------
     1. Download_Build: PASS * 
     2. Unzip: PASS * 
     3. Test_compile_log: PASS *
             No.of Compilation Warnings: 42636
 
     4. Test_CMworkarea_log: FAIL
             No.of Explicit Conflicts: 2
 
     5. Test_Install: PASS * 
     6. Test_Invokation: PASS * 
     7. Test_Deinstall: PASS * 
     8. Cleanup_Build: PASS 

           Passed : 7
           Failed : 1
           Total  : 8

 Product used for this test :  <product>.zip

 Note: * indicates mandatory TestCase.

          Build PASSED Sanity Test.

 Build Location: \\<share_name>\<Build_Share_Dir>


 Build Report: \\<share_name>\<Build_Share_Dir>\Build_Report.html


 Build Log Location: \\<share_name>\<Build_Share_Dir>\Build_Log


 Sanity Test Log Location: \\<share_name>\<Build_Share_Dir>\<product>.log

 *** AUTO GENERATED MAIL - Pls do not Reply***
 
 SDK Sanity Test Automation

The above test cases help us to determine the success or failure build.

How Software "Build Announcement" is useful?

Since the Build announcement and sanity testing is automated, it will happen over the night and everyone gets the build announcement before step in to office.

For Test Enginner: Successful Build Announcement makes sure that latest build can be taken forward for testing. So they can take it into test machines without worrying about the roll back and efforts spend to move to latest builds will not go waste.

For Developer: The test cases include analyzing log files and invoking the product. If both are fine, we can make sure that Compilation & Integration is successful. Developer will get to know the Integration status without waiting for testing results.

For Stakeholder: Build announcement is a Boolean type. It says, build is Successful or Failed. No need to look in to the dashboard and interpret multiple build job's status. It also helps to collects the metrics, for example how many broken builds in this particular sprint (in scrum).

Conclusion

Build announcement with a sanity test quality gate tests the builds even before it is announced (released) for consumption. It will improve quality because it finds issues as early as possible. It is not just analyzing the compiler log files but it actually tests the product and it will be helpful to find out integration issues.

More test cases can be added for improving the build success confidence level. Since the test cases are automated, it avoids the human errors and improves the quality.

Build Announcements helps to reduce the engineering effort and improve quality!