Having set up audio streaming with my RPi that was collecting dust (since my last post :), I ended up with a left-over RPi Zero W. And we finally decided to showcase all family photos, perfect timing! Setting up a networked photo frame has a couple of quirks to it…

I am starting new home automation project with RPi, and will try to cover some interesting aspects of this development here.

One of the features of my project is many-to-many streaming of video content on LAN. So to do a stress test of RPi i wanted to set up a simple stock (no custom code) environment to prove that it will work.

I am using omxplayer and VLC (on windows) to set this up: read more

I am starting new home automation project with RPi, and will try to cover some interesting aspects of this development here.

I am using Raspberry, Kivy and Python to draw most parts of the UI. And of course I want to use my workstation to actually develop the code in Visual Studio Code, while using RPi touchscreen to interact with the app.

First step is to set up a comfortable dev enviroment - press F5, and code runs to the breakpoint. Sounds easy - RPi has good python environment, python tools for visual studio are available, remote debugging is supported… Not so fast! There are about 2 tutorials on the web on how to set this up, and they won’t really cut it.

2013 Core i7 in HP Elitebook
2011 Core i7 in Lenovo Thinkpad W520
2011 Core i3 in custom build
2009 Core i7 in HP Elitebook
2009 Core 2 in Dell Studio 17
2005 Mobile Pentium 4 in HP ZD8000
2000 Athlon Classic in custom build
1998 SPARCstation 5 (Netra)
1996 Pentium P54C at 120Mhz in custom build
1995 80486 IntelDX4 at 100Mhz in custom build
1993 80286 at school lab
1991 8086 at school lab
1990 Electronika BK-0010.01
1989 Elektronika MK-52

Had memsql come by and give architecture presentation yesterday so i started reading http://blog.memsql.com/common-pitfalls-in-writing-lock-free-algorithms/ … complicated, and no detailed explanation of how it works anywhere! Kinda need to know your c/asm… and in the end, you get to things like “lock; cmpxchg16b” (mentioned in blog and here https://github.com/memsql/lockfree-bench/blob/0a82132d21d3779820720d5591f2b3bae9f6b65f/stack/boost/atomic/detail/gcc-x86.hpp#L1443 )

It’s a hardware assisted atomic operation (two of them – compare, and swap). Neat. There’s still cache sync overhead in hardware (actually pretty huge relatively speaking), but what can we do… I just wish this was explained better somewhere.

Also, as this article summarizes, http://moodycamel.com/blog/2013/a-fast-lock-free-queue-for-c++ “the cache coherence protocols and memory barrier instructions don’t seem to scale very well” – so shared-memory multi-CPU architectures will hit the wall too, and we’ll need new type of memory architecture to support in-memory databases. It will still lock, but better 🙂 In fact, the latest and greatest Intel CPU already got some instructions for this, http://en.wikipedia.org/wiki/Transactional_Synchronization_Extensions

Standing in the shower this morning I was thinking about the past and the future. How the world changed around me, and how I changed. About happiness and fulfilling life. Then I thought about my browser – hey, it’s the shower – anything goes.

Mosaic 2.0 – Hi Mozilla – 1995

My first exposure to the web happened via my dads work. They have purchased dial-up plan from one of first ISPs in Moscow – Glasnet. I am sure it cost something ridiculous.

I just got my second certificate for the two MongoDB courses I was interested in – Python and DBA.

I do highly suggest it to everybody – it’s relatively simple. I would say that DBA course is much simpler in the homework/exam. So the fact that I took time to study for Python course helped quite a bit with DBA course.

https://education.mongodb.com/

Keep in mind that as other courses I took ( https://www.coursera.org/ ) the courses with MongoDB are time-sensitive. You have to get some time to finish the assignments by 9PM EST on Monday. For most people this will mean spending 2-4 hours during the weekend. Also, since they are time-sensitive the courses don’t run all the time – you have to start on certain date.

But on the plus side, the forum is very lively while the course is running – both students and MongoDB support are answering questions and giving suggestions. That was a big help.

Also, general exposure to NoSQL ideas would help – your head may spin a little less. I really found it quite amusing that sharding was not major part of dev course – why else would you pick NoSQL?

Note: since writing this, the App Cache has been depricated in favor of Service Workers - fiddler-like interseptors that can implement caching among other things. While implementation is alot more flexible, it does need a bit of bootstraping, e.g. with help of Toolbox project

Cordova is great, right? Develop once, run everywhere etc etc. But what about updates? What about being backward compatible with all the new web services you are developing on the site, packaging the app and worrying about versions… no, we web developers can’t stand it.

First order of business – how to do updates. Common misconception is that traditional binary app stores (ios/android) don’t want the apps to self-update. It’s actually not true – you are allowed to self-update as long as the code you are updating runs in WebView. Bingo!

So today we got one of our arabic clients request – customize the order template to include the arabic text in the captions and text areas.

First thing i did was to try and just give FOP russian text – that just gets output as # signs. To get this working you need to follow this good old tutorial – it mainly talks about doing it manually (what i was doing first). The more extended, and newer (for versions 0.93 and 94) article is naturally located on official site. You will especially need it when you want to do it from code, not command line.

The font that you can use in Windows which covers almost all languages (per unicode) is located at c:\windows\fonts\arialuni.ttf . It’s a huge (22Mb) font that generates a pretty big font metrics (about 700k for me). This file, however, is not standard in windows – it seems to be part of the Microsoft Office installation. None of our windows servers have it by default, for example. Or you can just use plain arial, which covers all european scripts.

Finally tried doing arabic script using this font technique. This doesn’t work, and it doesn’t look like it will work any time soon. The problem of course is BIDI (bi-directional text support) – FOP can use arabic characters, but can not reverse the direction (and one can not simply reverse character order before giving it to FOP – arabic non-capital letters have to be connected to each other in a very special way). There’s no bidi support in FOP (although XSL-FO does have it in the spec). The mailing archive hints to some people working on it successfully, however the major obstacle seems to be lack of proper BIDI support in Java 1.3 itself – only 1.4 added needed methods, but FOP wants to stay 1.3-compatible.

Update: well, I just got a notification of 0.95 beta, and it does drop the 1.3 requirement. Good news! Hopefully the custom work that is done by other developers will be included eventually.

We had to revert to a workaround – the areas where arabic is needed are going to be bitmaps that our customers will be able to upload the images for.

“flows” in XSL-FO can have static areas surrounding them, which are naturaly used to make headers and footers on pages. Let’s consider sales invoice like this:

sample invoice

Here, of course, you can position header and footer absolutely using

<fo:block-container left="xx" top="xx" position="absolute">.

However, when your invoice grows and you run out of space on the page, and you want to see something like this:

invoice how it should page correctly

public static string SerializeObject(object obj)
{
 MemoryStream temp = new MemoryStream();
 XmlSerializer serializer = new XmlSerializer(obj.GetType());
 serializer.Serialize(temp,obj);

 temp.Seek(0,SeekOrigin.Begin);
 byte[] byteArray = new byte[temp.Length];
 int count = 0;
 while (count < temp.Length)
 {
  byteArray[count++] = Convert.ToByte(temp.ReadByte());
 }
 ASCIIEncoding asciiEncoding = new ASCIIEncoding();
 char[] charArray = new char[asciiEncoding.GetCharCount(byteArray, 0, count)];
 asciiEncoding.GetDecoder().GetChars(byteArray, 0, count, charArray, 0);
 return new string(charArray);
}
public static object DeSerializeObject(string s,object obj)
{
 ASCIIEncoding asciiEncoding = new ASCIIEncoding();
 byte[] inp = asciiEncoding.GetBytes(s);
 MemoryStream temp = new MemoryStream();
 XmlSerializer serializer = new XmlSerializer(obj.GetType());

 temp.Write(inp,0, inp.Length);
 temp.Seek(0,SeekOrigin.Begin);
 return serializer.Deserialize(temp);
}

Update: apparently XmlSerializer is pretty limited in which types it can reflect, it’s designed to fit current WDSL schema, which is limited itself. Well, not really, but in any case right now XmlSerializer can’t serialize multidimentional arrays, for example. In the code above you can replace XmlSerializer with SoapFormatter, while keeping everything else exactly the same. That will generate different (SOAP) envelope and you will be able to serialize any type of data.

When generating reports from XML data using XSLT I have to perform complex mathematical operations. Or, rather, operations that generic XPath can’t do. To achive that you can use variables in XSLT and fill node lists with them. So we have:

<xsl:variable name="totalCommission">
 <xsl:for-eachselect="orderz/accounts/payment/orderzs[@o_status != -9 and @o_prodtype != 'J']">
  <accum><xsl:value-of select="(@o_extprice  - @o_discount -itemmaster/itempricelist/@storecost*@qty) * ../../../../@salescommission div 100"/></accum>
 </xsl:for-each>
</xsl:variable>

as a result we create a variable that generates its element using some algorithm. After that, we can run a sum operation against it:

$<xsl:value-ofselect="format-number(sum(msxsl:node-set($totalCommission)/accum),'#,##0.00')" />

Very important in this operation is support of node set generator msxsl:node-set(), which is supported in MSXML that i use in this case. To use it the root XML node will look like this:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"version="1.0" xmlns:msxsl="urn:schemas-microsoft-com:xslt">