Screenshots of the day

a rough interface for the database, rather a rough index, a utility for the scheduler (more on this later).

Trying to figure out the whole system …

Posted in Uncategorized | Leave a comment

DESCRIBE images

Currently busy working on a sketch for the database. How to turn the informations we collected from the drives into a database backend?

At the moment a table Images that replicates the fields used by Michael when he produced the spreadsheet.

And a python script, walking again the drive and feeding its findings into the datastore.

While typing the code and thinking about the datastructure, I am listening to the lectures and Erkki’s stream of thoughts. Associative memory again.

Posted in Uncategorized | Leave a comment

Browsing Erkki’s diary with the help of Paul Hough

During a large part of his life Kurenniemi has produced a diary. Under different forms, written by hand in notebooks, on his computer, in audio cassettes and video. I recently opened a notebook exported in pdf from mathematica conceived by Stephen Wolfram. We can read Erkki’s stream of thoughts about maths, drawing, women, technology, sex … This notebook also contains hand drawings saved as vectors. In one entry, Erkki mentions freehand, but we don’t know if he refers to the program or to a free hand technique

Pdf is a very comfortable format for reading a text. To interpret it, to index its content, to perform manipulations on it, it is another story.
Several tools, like pdftotext allow to extract the textual content out of a pdf to make it searchable for instance.

Out of curiosity, I was looking for a way to detect in which pages of the pdf he had inserted images. Using the command pdftoimages, I extracted all pages of a pdf and saved them as bitmaps.  I decided to read the pdf in the company of opencv. Within Opencv, I used a function, cvHoughLines2, that allows to track optimally the different lines present in an image. This function is named after a patent introduced in 1962 by Paul Hough and later refined, improved and transformed several times before having the form we know today in computer vision.

Line detection

“In automated analysis of digital images, a subproblem often arises of detecting simple shapes, such as straight lines, circles or ellipses. In many cases an edge detector can be used as a pre-processing stage to obtain image points or image pixels that are on the desired curve in the image space. Due to imperfections in either the image data or the edge detector, however, there may be missing points or pixels on the desired curves as well as spatial deviations between the ideal line/circle/ellipse and the noisy edge points as they are obtained from the edge detector. For these reasons, it is often non-trivial to group the extracted edge features to an appropriate set of lines, circles or ellipses. The purpose of the Hough transform is to address this problem by making it possible to perform groupings of edge points into object candidates by performing an explicit voting procedure over a set of parameterized image objects (Shapiro and Stockman, 304).”
http://en.wikipedia.org/wiki/Hough_transform

The presence of non-horizontal lines give me a hint to select the pages with images. In the last pictures, the pages that only contain horzontal lines are discarded (strikethrough).

line detection and selection

Code

/* This is a standalone program. Pass an image name as a first parameter
of the program.  Switch between standard and probabilistic Hough transform
by changing "#if 1" to "#if 0" and back */
#include 
#include 
#include 
#include

using namespace std;

int main(int argc, char** argv)
{
    int linesdetected, linesthreshold;
    linesthreshold=3;
    linesdetected=0;
    IplImage* src;
    if( argc >1 && (src=cvLoadImage(argv[1], 0))!= 0)
    {
        IplImage* dst = cvCreateImage( cvGetSize(src), 8, 1 );
        IplImage* color_dst = cvCreateImage( cvGetSize(src), 8, 3 );
        CvMemStorage* storage = cvCreateMemStorage(0);
        CvSeq* lines = 0;
        int i;
        cvCanny( src, dst, 50, 200, 3 );
        cvCvtColor( dst, color_dst, CV_GRAY2BGR );
#if 0
        lines = cvHoughLines2( dst,
                               storage,
                               CV_HOUGH_STANDARD,
                               1,
                               CV_PI/180,
                               100,
                               0,
                               0 );

        for( i = 0; i < MIN(lines->total,100); i++ )
        {
            float* line = (float*)cvGetSeqElem(lines,i);
            float rho = line[0];
            float theta = line[1];
            CvPoint pt1, pt2;
            double a = cos(theta), b = sin(theta);
            double x0 = a*rho, y0 = b*rho;
            pt1.x = cvRound(x0 + 1000*(-b));
            pt1.y = cvRound(y0 + 1000*(a));
            pt2.x = cvRound(x0 - 1000*(-b));
            pt2.y = cvRound(y0 - 1000*(a));
            cvLine( color_dst, pt1, pt2, CV_RGB(255,0,0), 3, 8 );
        }
#else
        lines = cvHoughLines2( dst,
                               storage,
                               CV_HOUGH_PROBABILISTIC,
                               1,
                               CV_PI/180,
                               80,
                               30,
                               10 );
        for( i = 0; i < lines->total; i++ )
        {
            CvPoint* line = (CvPoint*)cvGetSeqElem(lines,i);
            int diffy;
            if(line[0].y>line[1].y){
                diffy=line[0].y-line[1].y;
            }else{
                diffy=line[1].y-line[0].y;
            }
            if(diffy>50){
            cvLine( color_dst, line[0], line[1], CV_RGB(255,255,0), 1, CV_AA);
            linesdetected++;
            //cout << "linesdetected: " << linesdetected <width;
        cross2.y=color_dst->height;
        cross3.x=color_dst->width;
        cross3.y=0;
        cross4.x=0;
        cross4.y=color_dst->height;
//        cout << "cross1 x: " << cross1.x << " cross1 y: " << cross1.y <
Posted in Uncategorized | Leave a comment

Motion vectors

MPEG movies, when compressed, analyze the movement of pixel data and use “motion vectors” to help achieve data compression. Movie players, like mplayer, support showing this data as a “debugging” feature.

This blog post discusses the mplayer feature, and here some C code is offered for doing the motion analysis.

Unfortunately, my mplayer crashed when trying to do this. Luckily I found some very useful instuctions for compiling mplayer from source on Ubuntu. This newly compiled mplayer did the trick!

Also, I tried to figure out how to see only the motion vectors (no image) but didn’t manage. The vector drawing feature is considered a “decoder option” and is as such tightly coupled with the decoding of the image data (it’s not a video filter). Eventually it might be possible to extract/isolate the motion data, though there seems reason for some doubt about the quality of the resulting data in general; I came across some discussions disputing the quality of this data.

Posted in Uncategorized | Leave a comment

More sorting by…

Tweaked the spreadsheet script to use PIL to resize the thumbnail images to a single pixel giving a rough impression of the average color of the image, then convert this to HSV for hue, saturation, and lightness values. Resulting images arranged by these values (horizontal position represents the value from low to high, vertical position is random):

by red…

green…

blue…

hue…

saturation…

value…

Posted in Uncategorized | Leave a comment

Faces of Kuvia

In our test folder Kuvia, we have many images taken daily by Erkki Kurenniemi. In order to detect which ones contain a human presence, we have run a  program written with Opencv using its python API. The algorithm that detects the faces tries to establish eyes alignement, the presence of the nose, the mouth and the hairline. The algorithm is our intermediary. Through it, we gain access to the faces but we also must accept its understanding somewhat extended of what a face is.

#!/usr/bin/python
import os,cv
facesdir="/home/nicolas/Public/kurenniemi-erkki/faces"
i=0
j=0
for root, dirs, files in os.walk("."):
    print root, "dirs", dirs
    if i>0:
        for f in files:
            print os.getcwd()
            #path = os.path.join(root, d)
            if f.endswith('JPG') or f.endswith('jpg'):
                imname=os.path.join(root,f)
                image=cv.LoadImage(imname, cv.CV_LOAD_IMAGE_COLOR) #Load the image
                hc = cv.Load("../faces/haarcascade_frontalface_alt2.xml")
                faces = cv.HaarDetectObjects(image, hc, cv.CreateMemStorage(), 1.2,2, cv.CV_HAAR_DO_CANNY_PRUNING, (220,220) )
                if faces:
                    for (x,y,w,h),n in faces:
                        cv.Rectangle(image, (x,y), (x+w,y+h), 255)
                        print imname,' x: ',x,' y: ',y,' width: ',w,' height: ',h
                        cv.SetImageROI(image, (x, y, w, h))
                        imtmp = cv.CreateImage(cv.GetSize(image),image.depth,image.nChannels)
                        imtmpname=facesdir+'/'+f+'-'+str(x)+'-'+str(y)+'-'+str(w)+'-'+str(h)+'.jpg'
                        cv.Copy(image,imtmp)
                        cv.SaveImage(imtmpname,imtmp)
                        cv.SetImageCOI(image, 0)

            j+=1
#    else:
#        break
    i+=1
Posted in Uncategorized | Leave a comment

Thumbnails by time

We start working on the contents of the folder “kuvia” — Finnish for images — and just one of several folders of images. Of the approximately 17000 images, 12000 have EXIF data (straight from a digital camera). I write a script (erkki_walk.py) to dump some of the metadata as tab-delimted text so that it can be viewed as a spreadsheet. In addition, thumbnail data is extracted to a separate (numerically indexed) file.

A second script (erkki_draw.py) reads the spreadsheet and draws a new single image (10000 x 1000 pixels) where the thumbnails are each drawn individually with horizontal position related to their timestamp of creation, and vertical position random:

Horizontal placement relates to camera focal length:

Horizontal placement relates to camera flash value:

Horizontal placement relates to exposure:

Horizontal placement relates to time of day (hour + minute from 0 to 24):

Horizontal placement relates to time of year (from january to december)

Posted in Uncategorized | Leave a comment

EXIF Thumbnails

Working with the pyexiv2 library, discovered that images from digital camera’s typically have an embedded thumbnail image, available via standard metadata (EXIF) that is included within the image itself. Work with thumbnail images generated by the camera is interestesting in that it helps speed the process of working with the bulk of images (no need to regenerate the thumbnail) and somehow honoring the actual data in that the thumbnail had been generated by the specific camera.

Posted in Uncategorized | Leave a comment

Associative Memory

While working on the image material in the digital archive, we listened to an audio recording of a lecture where Kurenniemi described the “associative memory” model of some of his early instruments where, in contrast with more conventional addressable memory, data are stored according to content-specific critieria (like the step position in a musical sequence) to allow a layering of musical attributes in time in a highly compact and efficient manner.

Wikipedia on “Content-addressable” memory

Posted in Uncategorized | Leave a comment

Computer Palette

Posted in Uncategorized | Tagged | Leave a comment