Flash card photo copy script

Something sorely missing in OSes and photo organization apps – a simple way to move your photos to hard drive and organize them by date. Here’s the script I’ve been using for couple of years, so it’s battle tested. This is looking for Canon camera folders on E: flash card, and using D: drive for storage – so adjust as necessary.
This is designed to keep your raw files separate from jpegs – I always shoot raw+jpeg or just jpeg, so this works well to de-duplicate my photo organization apps (lightroom and picasa).
The resulting folders are stored in d:\Raws and d:\Photos using the photo’s date as folder names.

@echo off
set "source=D:\Photos\TempTransfer\Incoming"
set "targetRoot=D:\Photos\TempTransfer\Outgoing"
set "targetRootRaw=D:\Photos\TempTransfer\OutgoingRaw"

robocopy E:\DCIM\100CANON\ "%source%" *.* /move
robocopy E:\DCIM\101CANON\ "%source%" *.* /move

for %%F in ("%source%\*") do (
  for /f "tokens=1,2,3 delims=/ " %%A in ("%%~tF") do (
    if "%%~xF" == ".CR2" (
        if not exist "%targetRootRaw%\%%C-%%A-%%B" mkdir "%targetRootRaw%\%%C-%%A-%%B"
        move "%%~fF" "%targetRootRaw%\%%C-%%A-%%B"
    ) else (
        if not exist "%targetRoot%\%%C-%%A-%%B" mkdir "%targetRoot%\%%C-%%A-%%B"
        move "%%~fF" "%targetRoot%\%%C-%%A-%%B"
    )
  )
)

robocopy /move /e "%targetRootRaw%" d:\Raws
robocopy /move /e "%targetRoot%" d:\Photos

My CPU history

2013 Core i7 in HP Elitebook
2011 Core i7 in Lenovo Thinkpad W520
2011 Core i3 in custom build
2009 Core i7 in HP Elitebook
2009 Core 2 in Dell Studio 17
2005 Mobile Pentium 4 in HP ZD8000
2000 Athlon Classic in custom build
1998 microSPARC in SPARCstation 5 (Netra)
1996 Pentium P54C at 120Mhz in custom build
1995 80486 IntelDX4 at 100Mhz in custom build
1993 80286 at school lab
1991 8086 at school lab
1990 K1801VM1 in Electronika BK-0010.01
1989 Elektronika MK-52

Memsql is lock free, but how?

Had memsql come by and give architecture presentation yesterday so i started reading http://blog.memsql.com/common-pitfalls-in-writing-lock-free-algorithms/ … complicated, and no detailed explanation of how it works anywhere! Kinda need to know your c/asm… and in the end, you get to things like “lock; cmpxchg16b” (mentioned in blog and here https://github.com/memsql/lockfree-bench/blob/0a82132d21d3779820720d5591f2b3bae9f6b65f/stack/boost/atomic/detail/gcc-x86.hpp#L1443 )

It’s a hardware assisted atomic operation (two of them – compare, and swap). Neat. There’s still cache sync overhead in hardware (actually pretty huge relatively speaking), but what can we do… I just wish this was explained better somewhere.

Also, as this article summarizes, http://moodycamel.com/blog/2013/a-fast-lock-free-queue-for-c++
“the cache coherence protocols and memory barrier instructions don’t seem to scale very well” – so shared-memory multi-CPU architectures will hit the wall too, and we’ll need new type of memory architecture to support in-memory databases. It will still lock, but better :)
In fact, the latest and greatest Intel CPU already got some instructions for this, http://en.wikipedia.org/wiki/Transactional_Synchronization_Extensions

JavaScript, wherefore art thou

Standing in the shower this morning I was thinking about the past and the future. How the world changed around me, and how I changed. About happiness and fulfilling life. Then I thought about my browser – hey, it’s the shower – anything goes.

Mosaic 2.0 – Hi Mozilla – 1995

My first exposure to the web happened via my dads work. They have purchased dial-up plan from one of first ISPs in Moscow – Glasnet. I am sure it cost something ridiculous.

Fun fact – this was my first exposure to 419 email scam too, back in 1995. Later I learned this has been going on via fax for years prior to that.

So what does a young person do back then – remember, no knowledge of web. Purchase a book of course – a crude translation of some “book about internet”. Naturally it talked about all valuable telnet, ftp and gopher resources.  And of course some web. Once I installed Mosaic the internet did turn into something visual and useful – it was awesome.  I remember sitting in the factory office, looking at some astronomy paper on the web. And clicking links! Over time I did switch to Netscape 2.0.

Chronologically, in 1994 almost 100% of my time was spent in MS-DOS developing in TurboPascal or Turbo C++ . Next year, Windows 95 comes out. So I jumped from the world of 286 (and sometimes even 8086) computers straight to the web.

Netscape JavaScript 1.0 book – 1996

In the summer of 1996 I started part-time work at ISA.ru. The first thing I was given was Sun’s Netra server – with all the glorious Solaris software (like CDE and Netscape Enterprise Server). I had zero knowledge of web development – but it was interesting that Sun included a couple of books on HTML and Netscape JavaScript.

This was an new experience reading these, of course – first reference book in English. What I still vividly remember what it actually covered – all the JavaScript could do back then is work with document.forms collection. Netscape 2.0 was the first version to start supporting JavaScript (first called LiveScript) back in 1995.

At this time, I also got a hold of my schools subscription of Wired magazines. Oh boy, was it cool back then. Wired basically invented the vivid, metallic and font-crazy layout. But of course it was the articles that you read the magazines for. It really opened my eyes to what was happening with the web.

IE 3.0 and Netscape 4.0 – 1997

Prior to 3.0, IE was basically licensed Mosaic.  Remember, Bill Gates didn’t believe in the Internet – MSN was the way of online.

Also, 3.0 was bundled with Win 95 SP2 (the good win 95, so called OSR2) so it started the first browser war and decline of Netscape. The important milestone for JavaScript here is introduction of JScript – actual support for JavaScript in Microsoft’s own product.

Netscape releases 4.0 (Communicator) 6 months later. To me, this was a major change – this became my main browser for years to come (until Mozilla). Lots of time spent actually developing web sites for it.

IE 4.0 – 1998

This same year, IE 4 is released. This was included in Windows 98 next year.

While I wasn’t aware of the internal changes at the time, in retrospect this is probably the main change in IE and web. It was called the Trident engine. In fact, it’s still used to date in IE on all platforms. It introduced proper DOM and DOM manipulation – which was really a horrible experience in Netscape.

In 1999, next version 5.0 included support for AJAX. So basically you could create modern web in 1999. Which in fact was done by OWA in 2000. The OWA team created the original xmlhttp ActiveX component, which was then included in IE 5.0 as Microsoft.XMLHTTP.

Layers vs divs… neither! – 1999

At that point the future of dynamic HTML was not clear. And everybody wanting to create a dynamic page had to support both IE’s DOM and Netscapes Layers. Not fun, as they work completely differently (and Layers was buggy).

The only cross-platform solution was to regenerate the whole page using document.clear() and document.write(). While killing your own javascript of course. So this only works with … frames! Javascript from one frame can overwrite HTML of another frame. That was the standard of dynamic pages back then – for example web chats used this technique.

Netscape 6 – 2000

Mozilla project is a rewrite of Netscape’s code. It has brand new Gecko engine, plus a ton of inherited code for all other parts of the browser (security, networking etc.). It reached the 0.6 – I was already trying it out as part of Mozilla Suite. And it was horribly broken.

At this point, I moved to US and started work as web developer – so I finally got exposed to all the crazy things in NN4. Gecko was a huge hope and breath of fresh air. But it simply wasn’t ready – AOL releasing production NN6 based on 0.6 of Gecko was an act of desperation.

IE 6.0 stagnation – 2001-2006

When IE 6.0 released with Windows XP,  Netscape was completely dead. IE effective share is 90%. Mozilla 1.0 is one year away. 

The web developers concentrated on supporting NN4 and developing for IE6. This is why we still have to deal with it – a huge amount of investment was made between 2001 and Mozilla finally taking some meaningful share of the market with Firefox in 2005.

The JavaScript and AJAX support is OK. But layout are horribly broken, as any CSS developer knows.

Mozilla 1.0 – let’s party? – 2002

When Mozilla finally released in 2002 I was so so happy. Even went to one of release parties in Cambridge, Mass. We thought that all we had to do now was wait for the adoption. Didn’t happen – why would anybody use some other browser? Remember, IE was completely dominant both for consumers and developers.

Phoenix, I mean Firebird … ok ok, Firefox! – 2003-2005

Changing name 3 times is not awesome for a project that has market share as its main goal, right? I mean come on, you are developers – pick some non-conflicting name.

I actually didn’t switch to Firefox until much later, when it became obvious Mozilla Suite wouldn’t get updated.

In 2004, once Firefox got off the ground, the www.spreadfirefox.com was launched – and it was awesome. We actually got a full page ad for Firefox in New York Times! https://blog.mozilla.org/community/2013/06/10/milestone-ad-for-firefox-in-new-york-times/

Finally in 2005 IEs market share began to erode. A combination of factors really affected that:

  • XP was unsecure with IE, and Vista failed in 2006. Security on the internet becomes a huge concern for users. Even Microsoft switches to firefighting mode with security push in 2002 and releases major architectural security changes in XP SP2 in 2004.
  • IE had no major releases until… wow, 2006! And realistically, for developers, until 2009 with 8.0.
  • Gecko finally caught up with IE if you can call it that. Sites actually rendered the same. Developers started to be more standards-aware. A good example is document.all support in Gecko – it’s an interesting way to work around the issue. You can’t check document.all for existence. But if you do document.all[“foo”] then you do get a result.
  • Web 2.0 starts – just in 2005 we saw Facebook and YouTube. Browsing has to become better – and Firefox delivers: speed, very useful addons, security.

KHTML – a what what? – 2003-2007

Apple (which just made iPods and Macs at that point) decides to make their own browser. And they don’t pick an easily customizable Mozilla for that, blasphemy! They picked some broken part of KDE for that.

Little did we know, they were working on web tablet since 2002. This was part of the project. Of course, eventually they decided to ship smaller factor first (iPhone).

4 years into the project, iOS finally ships. And Webkit instantly becomes best and fastest mobile browser. Poor Opera.

So it’s OK to build your own browser, Google Chrome steps in 2008

The main fear when Google started their own browser was the fate of Firefox. They were giving tons of money to Mozilla to have Google as default search engine at the time. 6 years later, they are still paying.

The innovation that Chrome brought was self-updating. After all, if the site can self-update why can’t the browser. Firefox, after fighting with old schoolers finally adopted the model too. And so did Microsoft, at least partially – IE now installs automatically as part of Windows Update. Previously it had it’s own terms you had to accept and was an optional update.

OMG, let’s catch up! IE9 Chakra – 2011

The browser world is moving like crazy – since 2007, there are Firefoxes, Chromes, Safaris and Operas eating into IE market share. Nice touch mobile browsing is growing. By the end of 2010, Internet Explorer is under 50%.

After playing around with .NET and Silverlight the writing is on the wall for Microsoft – they have to embrace the web. The old layout and javascript engines have to go. Browser has to update separate from OS and much more frequently. It has to work on mobile. Microsoft got to work in 2009.

As is traditional for Microsoft, when they invest into a rewrite they really over-invest – compared to gradual progression of other browsers. Everything is different:

  • Hardware rendering (requiring Vistas DirectX – so no XP compatibility)
  • New JITed JavaScript engine – very very fast
  • New layout engine – finally Acid support
  • New standards support – SVG, HTML5 video, canvas, WOFF

It helped lay the foundation which is still used for IE11 and up, with more standards and compatibility being added. And it’s a great mobile browser too. Windows 8 also includes an application runtime based on IE9 which has no browser restrictions – alas, that runtime is not present on Phone edition, so you have to keep using PhoneGap / Cordova.

Finally, 10 years later, new standards emerge

It’s not W3Cs fault – after all they simply represent the industry, they can’t force anything. But the standards that were being developed – XHTML and XHTML2 were made in vacuum, trying to make existing markup nicer. The world wasn’t interested in making web apps – XBAP, Silverlight and Flash were the future.

In 2004 a small group of developers actually interested in adding features to HTML formed, and started Web Hypertext Application Technology Working Group (WHAT-WG). Initially we mainly wanted to improve forms, so it was an alternative to W3C XForms standard. But eventually, this morphed into full-fledged next version of HTML. A practical HTML spec actually useful for making the apps.

It was approved by W3C in 2007 and could not have happened if IE still had 90% market share. By 2007 it actually had vendor support (Mozilla, Apple and Opera).

So finally in 2011, 10 years after IE took over, developers actually got new APIs to use.

Now

This is hard topic to talk about. But 2 years into web app revolution, we are only starting to scratch the surface of “normal” application development. Chrome Development Tools are becoming a part of IDE. Easy mobile debugging only starting to show up. Profiling tools are finally maturing in Chrome and IE11. Automated testing, especially on mobile, is still painful but getting better.

The great story is the frameworks and libraries. Between polyfills, OR/M, MVC, graphical libraries, packaging, unit testing, and even server-side javascript – it’s hard to pick what’s more awesome.  My favorite success story is Sencha which went from single person GPL project to a 150 person company doing nothing but javascript framwork.

A look into next 10 years

Mobile will be majority of browsing – it already is depending on where and how you count. For example, many very popular apps use browser component. Additionally, mobile share is extremely high outside of US. On the business front this may take all of 10 years, but already the business software that is currently being developed is all web.  Every time the market share conversation starts, people talk about business/desktop segment – and sure, there will be emulators to run the current non-web apps. Same as we have emulators, and sometimes even wrappers, to run terminal apps.

Web is finally a standard now – a standard in a sense that every platform has to do it equally well. This sets a very high bar for entry – so making a new mobile OS is hard. Previously, you could embed WebKit easily – the way Blackberry has done it. However, with Blink branching, it is highly likely that WebKit will become Apple-only. They can’t close-source it, but they can make it un-portable. Same with Blink. The poor Mobile Firefox is limited to Android, but Chrome already ships with OS.

This leaves us with triumvirate of IOS WebKit, Android Blink and Windows IE. With Android becoming the windows of mobile (by market share), Blink will be the web development platform. It is the new IE. There are of course differences – the update cycle on phones is pretty fast, Google likes to push web standards. We may even see Blink self-updating on android. It looks like the turbulent period of having many browsers is coming to an end. The web apps in the next 10 years will concentrate on supporting Blink and polyfilling into WebKit and IE.

What innovation can then happen in the browser space? Thanks to Chrome the UI of the browsers already went away, and so did the support for additional functionality like RSS. The innovation will be all under the hood, invisible to consumers. Browser simply became the platform for development, similar to Flash or Silverlight – and these hardly had any UI.

What can we expect? Short term is obvious – all browsers getting support for all current standards like WebRTC and some upcoming like web components. Long term, we will get new standards for everything previously invented. After all it’s a platform, and you need to do everything – like memory management, raw networking, hardware access, and multi-threading. Maybe even a different/better language. Maybe even bytecode support  for JavaScript.

Really, pretty boring stuff. But boring is good – I would rather concentrate on making the platform much easier to develop for. And here we finally come to critical point – hey, we have a single platform? For the first time in computing, we have a single connected platform to develop apps for? Apps that can run on any size screen, and any type of device?  The unfulfilled promise of the past? This platform self-assembled and evolved from the horrors of the browser wars.

For the next 10 years, we welcome our Blink overlords.

MongoDB free training

I just got my second certificate for the two MongoDB courses I was interested in – Python and DBA.

I do highly suggest it to everybody – it’s relatively simple. I would say that DBA course is much simpler in the homework/exam. So the fact that I took time to study for Python course helped quite a bit with DBA course.

https://education.mongodb.com/

Keep in mind that as other courses I took ( https://www.coursera.org/ ) the courses with MongoDB are time-sensitive. You have to get some time to finish the assignments by 9PM EST on Monday. For most people this will mean spending 2-4 hours during the weekend. Also, since they are time-sensitive  the courses don’t run all the time – you have to start on certain date.

But on the plus side, the forum is very lively while the course is running – both students and MongoDB support are answering questions and giving suggestions. That was a big help.

Also, general exposure to NoSQL ideas would help – your head may spin a little less.  I really found it quite amusing that sharding was not major part of dev course – why else would you pick NoSQL?

Mongo DBA Certificate Mongo Developer Certificate Mongo Courses

Phonegap / Cordova + AppCache

Cordova is great, right? Develop once, run everywhere etc etc. But what about updates? What about being backward compatible with all the new web services you are developing on the site, packaging the app and worrying about versions… no, we web developers can’t stand it.

First order of business – how to do updates. Common misconception is that traditional binary app stores (ios/android) don’t want the apps to self-update. It’s actually not true – you are allowed to self-update as long as the code you are updating runs in WebView. Bingo!

There are solutions on the market that give you such self-update mechanism, for example WorkLight that was recently acquired by IBM. It’s really not terribly hard to write this yourself if all you need is updates – get a zip file with all html/css/whatnot, download when application starts, extract, then actually render the UI.

However, there exists a system that is designed to do just that, and it’s arguably pretty good at that. The support for AppCache is very good both on modern desktop and mobile. The only question is how you’d use AppCache with Cordova.

Well, obviously your app can be a web site. Or web site can be the app… It’s really the same for web developers. I call most of the sites I create applications, because they are applications, from the customer point of view. Plus they can be pinned / clipped / made into an icon

I Steps on your site
1) Make sure you have offline manifest on the page that will be opened by Cordova
2) Double check – is there a redirect from the URL you think you are opening and actual URL that opens up? Appcache doesn’t work with redirects, so you have to have correct URL. For example, i had http://localhost/offline redirect to http://localhost/offline/ – and it didn’t work.
IIa Cordova iOS steps:
2) Allow Cordova to go out on the internet. In your project, go to Resources/Cordova.plist . Add record to External Hosts with your server name. You can just put star (*), but if someone makes your side redirect elsewhere you are in trouble – so only use for development.
3) Make sure cordova opens links inside its own webview, instead of safari, by changing OpenAllWhitelistURLsInWebView property to Yes.
IIb Cordova Android steps:
2) Allow Cordova to go out on the internet. In your project, open res/xml/config.xml and uncomment  <access origin=”.*”/> (this is only for development, see note in step 2 for iOS)
3) Add <preference name=”stay-in-webview” value=”true” /> to stay inside cordova
4) Adjust Android Cordova to enable AppCache. For some strange reason, it is not enabled by default for Android (while local storage and SQLLite are) – see setup() method in CordovaWebView.java file. It looks like in iOS UIWebView has it on by default and doesn’t let you mess with it. To fix this, just go into the only java file you should have in your app, under src/your.name.space/CordovaActivity.java (your file name could be different) and adjust the onCreate method to be like so:
public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        super.init(); // Initialize appView, since it's only initialized inside loadUrl by default
        android.webkit.WebSettings settings = super.appView.getSettings(); // get WebView settings
        String appCachePath = this.getCacheDir().getAbsolutePath(); // Set path to appcache
        settings.setAppCachePath(appCachePath);
        settings.setAllowFileAccess(true); // Let browser write files - doesn't work without this
        settings.setAppCacheEnabled(true); // Enable app cache
        super.loadUrl("file:///android_asset/www/index.html");
}
Update June 4 2013: Per comments, this is fixed in Phonegap 2.5, so code above in #4 is not needed anymore (i.e. leave it alone the way cordova generates it, don’t modify)
III Finally, the Cordova app
This one is very simple, as you might imagine. In www/index.html all you need to have is:
<!DOCTYPE html>
<script type="text/javascript" charset="utf-8" src="cordova-2.0.0.js"></script>
<script>
    function onBodyLoad()
    {        
        document.addEventListener("deviceready", function(){
            location.href = "http://192.168.2.108/offline/";
        }, false);
    }
    </script>
<body onload="onBodyLoad()"></body>
Replace the URL with your local testing site that you can start/stop to test how offline is working, or with any other URL.
That’s it, start the app!

Actually using Cordova APIs

One caveat is that at least on Apple AppStore the wrapped  site has to be more than site. It has to have some additional functionality that is not available via the web. And that’s where Cordova APIs come to the rescue, with access to all the interesting APIs a regular WebView/UIWebView doesn’t have access.

The first thing you have to do is detect that web site was opened from inside Cordova. It doesn’t look like there’s currently an API in cordova.js to tell us it loaded inside the Cordova web view. The only semi-working solution is to detect that onDeviceReady didn’t fire – but you have to wait to determine this fact, and that will delay your page loading in regular browser. Easier way is to pass some querystring parameter, url?fromcordova=true , and then include javascript and execute ondeviceready handler based on this parameter.

Second, cordova javascript actually has two versions – one for iOS and one for Android (and I am assuming others for other platforms). File name is same, content is different. The reason is that on iOS it’s using custom URL handler gap:// to communicate with Objective-C side. On Android, it’s overriding exec, prompt and alert methods in JavaScript and then on Java side in DroidGap class it’s intercepting these calls and actually using them to interact with JavaScript side (especially prompt, that can return data back to JavaScript). So the platform ID has to be another parameter in your URL, for example: url?fromcordova=true&platform=iOS

 Note: the above setup instructions are for Cordova 2.0.0 – in previous versions some things still had PhoneGap names.

My experience with ExtJS

So more than a year ago I wrote about my new project – starting a re-write of our main application using ExtJS, AJAX and web services.

The project actually progressed very well (albeit I was the only developer on it, but that’s even more fun!). I have a proof of concept application, window and class framework to build on, a set of business classes and complete database schema for a good Point-of-Sale application. I have proved that a RAD environment can be set up and extended using the technologies I picked.

The modules I have created are:

  • Customer and item entry and search (with cool use of LiveGrid extension to view entire inventory or customer base)
  • Order search, creation, modification and saving
  • Sidebar for most important features
  • Theme changing (hey, it has to look cool)
  • Login and security system (using asp.net membership and roles on the server)

So far that’s all, I have actually stopped about 14 months ago, and started working on brand new project (more on that later).

I still think ExtJS is a very good framework, especially if you want to design a true AJAX app (not just spice up your old one). With all the FUD that was created by license switch, it’s hard to see the forest through all the trees. Yes, license changed, yes it was done poorly. But face it – it doesn’t make ExtJS any less cool as a technology, and it is worth the money they are asking (in my opinion anyways).

Word 2007 as blogging software

Microsoft Live Writer, cool software. An oddity of sorts for Microsoft – something that works well in v1. And works with multiple blog providers. And looks nice while doing that (with theme download and a bunch of addons).

But I can’t use it. I have a 130dpi monitor (one of those 1920×1200 17″ laptops – calculate your dpi here http://members.ping.de/~sven/dpi.html ), and of course everything is tiny by default. Windows 7 and all apps I use do a very good job scaling their UI, finally. Well, maybe it was the same with Vista, but I didn’t have high dpi. With NoSquint for Firefox (IE8 zooms in by default, based on Windows scale settings – nice touch) and all fonts in Visual Studio adjusted I really can’t tell difference between normal 80-90dpi and my 133dpi – it just looks much better (more pixels + ClearType). In some other software I use I also had to adjust font sizes (Trillian Astra, Skype, Outlook). Can’t adjust anything in Writer – I tried everything, even found where it downloaded my blog’s theme and tried changing size there – doesn’t work, well no obvious place to change it anyways.

So today I finally decided to write something in my blog *crickets*, and trying Word 2007 now. Setup wasn’t without a problem, as Word doesn’t include the nice auto-configuration setup of Writer. For those interested, /yourmtfolder/mt-xmlrpc.cgi is the path to the MetaWebLogApi script of MovableType. Couldn’t make Atom work for some reason, although apparently there is mt-atom.cgi.

As a UI, Word is definitely a step back from Writer – no themes, no addons, categories/tags don’t download. But you know what, I don’t that much care – I have good size font to write in, and that’s all I need. Will see how it handles pictures and VS formatting next.

New ExtJS + ASP.NET AJAX project

Being an SaaS company from the start (web technologies were chosen to drive our main product), it only made sense for us to eventually completely embrace AJAX. In addition, the original application is basically ASP.NET 1.1, with an inconsistent graphical design. So sprinkling UpdatePanel on top would just add some dynamical elements to an outdated UI design, without any real noticeable changes to the user.

This, among other things, was the reason to go all the way and redo our application using a better UI library – naturally ExtJS was my first choice – I didn’t want to invest too much into proof-of-concept application, or, frankly, lock us into one of proprietary component libraries. Version 2.0 just came out, and their component model is quite mature to build an enterprise LOB (line of business) application on top of it.

At the same time I want to keep some standards in place, and chose to use ASP.NET AJAX for the server side, in the web services area. That way we are developing same web service for our AJAX JSON application and for our SOAP integration, using same code in ASP.NET web service. In fact, it works transparently most of the time, and same service code can be accessed from both types of clients. One other thing it adds is nice JavaScript proxy object support on the client side. In the end, while we are developing the application, we get automatic web services API already developed for our partners.

Thirdly, to cut down on development time, an OR/M is used. Currently it’s CodeSmith’s .netTiers template – which automatically generates DAL, BOL and even "service" layer. More importantly, it creates web service (asmx) file, which can be used as-is, or copied from.

The database backend is MS SQL 2005, as always. However, I made some use of xml column type, based on previous experience, to try and accommodate some clients unique requests, which don’t mandate a separate table schema change. The end result is that the product is generic enough to accommodate different scenarios, and fast enough with little overhead from all the XML parsing – it is only needed when a feature sees minor use in real life (and yes, I do know about xml indexing, but for inserting/updating some overhead is still incurred).

The resulting tiers look like this:

  • ExtJS UI
  • client-side code
  • ASP.NET AJAX JavaScript proxy classes
  • JSON data transfer
  • ASP.NET web service
  • CodeSmith .netTiers business and data objects
  • MS SQL database

 

I am very careful to re-evaluate each of the technologies used in this list – an experienced developer may question each and every one of the choices. There are very good (and possibly better) replacements for ExtJS, ASP.NET AJAX JSON, ASP.NET web services, or .netTiers. But by taking the possible changes into account, it’s not that hard to design a system where all these things can be swapped later on. The only real investment that is made, is made into UI code and database designs. So, for example, any code that we custom-develop on top of OR/M can we migrated to a different OR/M solution – and the objects/field names are still driven by initial database design.

But things have to progress, so while designing for change, we move forward as quick as we can! That’s one of XP mottos – programming only what is needed today, and implementing it as simply as possible. And not being afraid to change later on.

So with that, I’ll finish this introduction into our next generation product, and get back to work :) There are tons of things to learn in all that new tech, and I will try to write about it – most of the user-created code can be found in ExtJS forums, and really all it takes is integration.

Multilingual Apache FOP

So today we got one of our arabic clients request – customize the order template to include the arabic text in the captions and text areas.

First thing i did was to try and just give FOP russian text – that just gets output as # signs. To get this working you need to follow this good old tutorial – it mainly talks about doing it manually (what i was doing first). The more extended, and newer (for versions 0.93 and 94) article is naturally located on official site. You will especially need it when you want to do it from code, not command line.

The font that you can use in Windows which covers almost all languages (per unicode) is located at c:\windows\fonts\arialuni.ttf . It’s a huge (22Mb) font that generates a pretty big font metrics (about 700k for me). This file, however, is not standard in windows – it seems to be part of the Microsoft Office installation. None of our windows servers have it by default, for example. Or you can just use plain arial, which covers all european scripts.

Finally tried doing arabic script using this font technique. This doesn’t work, and it doesn’t look like it will work any time soon. The problem of course is BIDI (bi-directional text support) – FOP can use arabic characters, but can not reverse the direction (and one can not simply reverse character order before giving it to FOP – arabic non-capital letters have to be connected to each other in a very special way). There’s no bidi support in FOP (although XSL-FO does have it in the spec). The mailing archive hints to some people working on it successfully, however the major obstacle seems to be lack of proper BIDI support in Java 1.3 itself – only 1.4 added needed methods, but FOP wants to stay 1.3-compatible.

Update: well, I just got a notification of 0.95 beta, and it does drop the 1.3 requirement. Good news! Hopefully the custom work that is done by other developers will be included eventually.

We had to revert to a workaround – the areas where arabic is needed are going to be bitmaps that our customers will be able to upload the images for.