Pages

Sunday, 9 March 2014

Led strip control with Java

I’ve just ported a Python library to drive LPD8806 Led strips to Java.
The only dependency of the library is Pi4j.
The led strip I used was the Digital RGB LED Weatherproof Strip from Adafruit.
I’ve also made this library available on Github: https://github.com/glnds/LPD8806-Java. See Readme for me details

Saturday, 8 March 2014

Pi4J for Arch Linux

Some time ago I used Pi4J in one of my Raspberry Pi projects running Raspian. While trying to migrate this project to Arch Linux recently, I was unable to find a Pi4J version for that distribution. In the end I got it running by making a Pi4J version for Arch Linux myself. This tutorial describes how to build Pi4J on Arch.

If you don’t want to build the library yourself, you can just download the build here.

The installation instructions are at the bottom of this article and in the README on Github.

How to build Pi4J on Arch Linux

  1. Prepare your Raspberry Pi (Running Arch linux):
    > sudo pacman -S git-core make gcc libtool jdk7-openjdk
    
  2. Install WiringPi on your Raspberry Pi:
    > git clone git://git.drogon.net/wiringPi
    > cd wiringPi
    > ./build
    
  3. Test the WiringPi install:
    > gpio -v
    > gpio readall
    
  4. Set JAVA_HOME on your default profile:
    > sudo vim  /etc/profile
    
    export JAVA_HOME=/usr/lib/jvm/java-7-openjdk
    
  5. Set JAVA_HOME for your default environment. This is needed when Maven will connect with the Raspberry Pi over SSH.
    > sudo vim  /etc/environment
    
    export JAVA_HOME=/usr/lib/jvm/java-7-openjdk
    
  6. Clone the Github Pi4j Arch project:
    > git clone https://github.com/glnds/pi4j-arch
    
  7. Adjust the pom.xml
    > cd pi4j-arch
    > vim pox.xml
    
    ...
    <!-- DEFAULT RASPBERRY PI PROPERTIES -->
    <pi.host>192.168.1.1</pi.host>
    <pi.port>22</pi.port>
    <pi.user>root</pi.user>
    <pi.password>root</pi.password>        
    ...
    
  8. Install Maven: http://maven.apache.org/
  9. Build Pi4J:
    mvn clean install -P native -P hard-float
    
Important note: currently, the script only works when executed by the root user. If you want to run it under a different user, you will have to grant the user root privileges and suppress its need to provide a password. I’ve not tested it but probable you have to setup your user like this:
> sudo visudo

...
user    ALL=(ALL) NOPASSWD: ALL
...
You will also have to do some minor adjustments to the build.xml file.

This build procedure is based on the original build instruction of Pi4J.

Install Pi4j on Arch Linux

  1. If not yet downloaded, download the Arch build here (or use your own build).
  2. Copy the Pi4J library to your Raspberry pi:
    > scp pi4j-0.0.5.tar.gz pi@rpi.local:~/
    
  3. Extract the archive:
    > tar -xvzf pi4j-0.0.5.tar.gz
    
  4. Install the libraries:
    > sudo mkdir /opt/pi4j
    > sudo mv ~/pi4j-0.0.5/lib/ /opt/pi4j/
    > sudo mv ~/pi4j-0.0.5/examples/ /opt/pi4j/
    
Done! You can now use Pi4J on your Raspberry Pi running Arch Linux. Here’s an example:
sudo java -cp SomeJar.jar:.:classes:/opt/pi4j/lib/'*' be.pixxis.Example

Sunday, 2 March 2014

How to Integrate Elasticsearch into your application

If you want to integrate a state of the art search engine into your application, Elasticsearch is an excellent choice. Out of the box it offers you the most amazing search options, the REST API is well documented, easy to understand and there is vibrant community behind it to help you out if needed.

After you finished the setup, you can start with the integration. In regard to this, different strategies are possible. The one described here is very easy and will work for almost any application. I’m also convinced that this solution is best suited for integration in any legacy application.
Here’s an overview of the architecture:


As you notice, this architecture is based on a web application, but as you read on, you’ll notice that this approach works for other types of applications as well.

General idea

In most environments a search will only cover a subset of the data model, and within this subset your interest regarding search will consist a limited set of properties. Secondly, you want the search integration to have a minimal impact on any existing code/application.

Architecture

In the proposed architecture the general idea is achieved by creating a lightweight copy of the original object, this lightweight object is then stored in Elasticsearch.

The result in Elasticsearch is an index for every object collection you want to search on. Every index will contain a copy of the object but with its properties limited to the fields you want to search on. It is key to include the object’s identifier or primary key in the ElasticSearch copy.

Note: It is possible that you have to add extra fields to the lightweight object besides the ones you want to search on, this is the case if the search fields don’t include fields needed to display a proper result.
An example: - Object:
Movie: {
    Id,
    title,
    description,
    release date,
    producer,
    actors[],
    tags[],
    category,
    duration,
    language,
    subtitels
}
lightweight copy of the object in Elasticsearch:
Movie: {
    Id,
    title,
    description,
    tags[]
}

Synchronisation

In the proposed architecture it is a necessity that both collections are always in sync. You could easily achieve this by extending your data layer as follow:

  • On create: add a copy of the object to Elasticsearch. 
  • On update: check if the updated fields get persisted in Elasticsearch as well, if so, update Elasticsearch. 
  • On delete: remove the copy from Elasticsearch.

Note: for existing applications you’ll need to create a script to initialise Elasticsearch.

The final step is to redirect your application’s search queries to Elasticsearch’s RESP API. The result: you’ve enhanced your application’s search with all possibilities offered by elasticsearch.
If a user launches a search query in your application Elasticsearch will kick in from the background. The REST calls initiated from the application to Elasticsearch will return collections of lightweight objects. These collections can be returned to the user without further processing, so there will be no extra performance cost for search queries.

As soon as the user navigates through the result you’ll have the possibility to retrieve the original object from the database through its identifier.

Best practices:

  • Depending your requirements you could do the writes (for syncing) to Elasticsearch asynchronous, this way you don’t add extra lag to your application
  • In a setup like this it is really nice to have a data-pump script to (re-)create all your indexes from your database. It will make your life easy if you need to make an adjustment to your data model or if you need extra properties in your index. It will also be a life saving fallback in case your index gets out of sync or if anything goes wrong on your Elasticsearch cluster.
  • Use a tool to construct your Elasticsearch queries. From a Java background I could advise using Velocity for this purpose.

Advantages

  • The main advantage of this approach is the low impact on existing code and infrastructure.
  • It also offers a nice level of separation between your main and search functionality, so you could easily scale and balance your search separately from the rest of your application.
  • Easy to integrate.
  • Nice performance both on writes and reads.

Disadvantages

  • Storage overhead due to data duplication.
  • Synchronisation risks, if the synchronisation to Elasticsearch fails you end up with corrupted data in your index.

Saturday, 1 March 2014

Raspberry Pi: resizing the SD card root partition on Linux Arch

Do this only right after flashing the Arch image onto the SD-Card so if anything goes wrong you do not lose any data.

fdisk /dev/mmcblk0

In fdisk:

  1. Press ‘p’ to print the partition table:

    Command (m for help): p
    Disk /dev/mmcblk0: 14.9 GiB, 15931539456 bytes, 31116288 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x417ee54b
    
    Device         Boot     Start       End  Blocks  Id System
    /dev/mmcblk0p1           2048    186367   92160   c W95 FAT32 (LBA)
    /dev/mmcblk0p2         186368   3667967 1740800   5 Extended
    /dev/mmcblk0p5         188416   3667967 1739776  83 Linux
    
  2. Press 'd' to delete a partition:
    • Enter '2' to choose the second partition.
  3. Press 'n' to create a new partition:
    • Enter 'e' to select extended as partition type.
    • Partition number: <enter> to accept 2 as default partition number.
    • First sector: <enter> to accept default.
    • Last sector: <enter> to accept default.
  4. Press 'n' to create a new partition:
    • Enter 'l' to select logical as partition type.
    • First sector: <enter> to accept default.
    • Last sector: <enter> to accept default.
  5. Press 'w' to write the new partition table.

Reboot to force the kernel to recognise the new partition table.

sudo reboot

After the system has come back up you still have to resize the partition.

resize2fs /dev/mmcblk0p5

output:

resize2fs 1.42.8 (20-Jun-2013)
Filesystem at /dev/mmcblk0p5 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/mmcblk0p5 is now 3865984 blocks long.

Done!