Categories
Ideas

Media Micro Payments

Given the proliferation of payed media services like Netflix, Spotify or newspaper paywalls like the Times or Telegraph if there was a system where individual media items could be read for a small fee would people be interested in say paying 1 cent for an article or 10 cents to listen to a piece of music once?

However, micro-payment infrastructure is non-existent. Yes there are things like Patreon which serve artists and there are plenty of advertising driven services principally Google and to a lesser extent targeted ads on Facebook. The paid services such as Patreon expect ongoing payments which is annoying and mostly they are not micro whereas the advertising driven services require millions of views to generate a good income, for instance Spotify play return is 1:1000 that is 1 million plays result in $1000 and are biased to the favour big artists. But often most people listen / read / watch something once or at most twice so why not have a service where you consume per view say $0.10 or less. The service could be additive. That is if you consume the same item you ‘buy’ lifetime ownership.

The service would need to be a plugin or a wrapper website and the user authenticated and possibly anonymous to avoid irritating follow up. But if it was as simple as clicking a button to unlock an item with the cost clearly shown it could open up a lot of media that is otherwise hidden behind paywalls.

Categories
Agile

Is Conway’s Law Flawed?

Is Conway’s Law Flawed?

What is Conway’s Law

Conway’s Law says “organisations which design systems … are constrained to produce designs which are copies of the communication structures of these organisations.” . Also restated by Eric Raymond as “the organisation of the software and the organisation of the software team will be congruent” [1]. To further illustrate Conway’s Law, Conway states, “It is a consequence of the fact that two software modules A and B cannot interface correctly with each other unless the designer and implementer of A communicates with the designer and implementer of B. Thus the interface structure of a software system necessarily will show a congruence with the social structure of the organisation that produced it.” [2]
In many cases this makes sense, so you hear about software development teams aligned to a business unit. In banking, for instance, if the bank trades FX there will be developers working on FX trading apps but in addition there may be developers aligned to Finance who happen to use the rates supplied by the FX team. Consequently there will be an interface between these teams, even if its not direct. In fact teams that are not business aligned or form part of the organisational communicate usually get into trouble as they are not serving the organisations needs.

Is Conway’s rule a Power “Law”


  1. https://en.wikipedia.org/wiki/Conway%27s_law
  2. http://www.melconway.com/Home/Conways_Law.html
Categories
Blog Low Latency Technology

Peer 2 Peer Trading Demo

            SMR is building a peer 2 peer trading platform for trading on foreign exchange. This video shows two traders buy and selling from each other. For technically minded the app is built as an AngularJS single page app with web socket REST API Scala / Java server with Cassandra as the data store.

 

You can also see in on YouTube

 

Categories
Blog

Design Patterns

            Design Patterns
Pattern Description Java Usage
Constructional
Factory Method
Builder String Builder
Prototype
Singleton
Structural
Adapter
Decorator
Façade
Fly Weight
Behavioural
Command
Interpreter
Iterator Iterator
Observer Observer Listener
Pub Sub
State
Strategy
Visistor
Concurrency
Active Object
Double checked locking
Join
Lock
Monitor
Block Chain
Volatile
Compare and Swap

 

 

Categories
Java Technology

Java Tuning and Trouble Shooting

            <h2>Useful JVM Settings</h2>

-Xmx512m (maximum Heap space) -Xms1024m (minimum Heap size)
starting points 2G 32 bit 3-4G 64 bit and 1:3 young vs old
-XX:+HeapDumpOnOutOfMemoryError

Java Trouble Shooting Tools

  • heap dump
    • jmap -dump:file=filename.bin [pid]
      • hprof binary format
      • needs to be analysed in another took like MAT
      • or jhat to see in a web browser on port 7000 on local host
        • jhat -J-mx2000m filename.bin
    • jmap -histo [pid]
      • histogram of java object heap
      • quick text output
    • jmap -heap [pid]
      • to print java heap summary
  • thread dump / thread stack
    • jstack [pid]
  • enabling gc logs
    • -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:[logname]
  • visualJVM
  • jconsole
  • list open files for process
    • sudo lsof -p [pid]
  • list max open file limit
    • ulimit -S -n
Categories
Technology

Working Example C++ Addin for Excel on Mac OS X using CDT

            <h2>Introduction</h2>

There are some tutorials that try to explain how to create an Excel Addin for Mac OS X but none of the give you a working example for the Mac, and most addin tutorials are aimed at Windows. So I have created a working example which you can find here. This short article gives a bit more explanation on what was needed to make the example work.

Note. These instructions only work for Mac OS X.

Prerequisites and Installation

Install CDT (I didn’t try installing the Eclipse CDT plugin but apparently it works too). I used CDT Mars Release (4.5.0).

  • Download site: https://eclipse.org/cdt/downloads.php
    • standalone: http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers/marsr
    • update site: http://download.eclipse.org/tools/cdt/releases/8.7

Install Xcode, if not already installed. I used Version 6.4 (6E35b) with all updates applied.

Buy and install Excel Mac 2011, if not already installed.

Start Eclipse CDT and follow the instructions in README.md to import three projects into Eclipse. You should see this when you have finished:

cdt-projects

The C++ Addin – MacExampleAddin

The addin justs adds two doubles together. Its declared in addin.hpp and defined in addin.cpp. In fact we are not using C++ directly but we still use C++ file naming conventions which makes the compiler think we are using C++.

The project builds a 32 bit dylib and was created by choosing the Shared Library -> Empty Project  -> MacOSX GCC in the new Eclipse C++ project dialogue box. Important: You must build a 32 bit shared library for Excel 2011 which is a 32 bit app. So some changes were made to the settings (Eclipse menu -> Project -> Properties) as follows :

  • In C/C++ Build -> Settings Tab make the following changes:
    • MacOS X C++ Linker -> clang++
      • Dialect -> ISO C++11 (-std=c++0x)
      • Miscellaneous  -> -arch i386 -fvisibility=hidden
    • GCC C++ Complier -> clang++
      • Dialect -> ISO C++11 (-std=c++0x)
      • Miscellaneous  -> -c -fmessage-length=0 -fvisibility=hidden -arch i386
    • GCC C Complier -> clang
      • Dialect -> ISO C11 (-std=c11)
      • Miscellaneous  -> -c -fmessage-length=0 -fvisibility=hidden -arch i386

One of my objectives was to use C++11 but I couldn’t be bothered downloading the gcc compilers so I tried clang which is part of Xcode. After a bit of research I discovered the sensible people that wrote clang made the options the same as gcc so that either compiler can be used in place of the other. Eclipse CDT is more gcc focused so doesn’t know about clang which is why the settings changes below were made. As you’ll see, if you try this approach, clang works very well.

If the settings don’t work for you either I’ve missed something in the write up here so check the settings in Eclipse or time has marched on and versions have changed and this tutorial no longer works 🙁

addin.hpp

I declared the functions in an extern “C” directive to ensure the compiler doesn’t mangle the names as it will due to the hpp extension. This will make life simpler when we actually use the addin in Excel as the names in the header will be the same in the shared library. There are actually two functions on the header: addNum and addNum2 which will be explained in a moment but they do the same thing, add two numbers together.

addin.cpp

The file defines EXPORT which is a compiler/linker directive that allows us to selectively export functions in the library along with the appropriate compiler / linker options, that is “-fvisibility=hidden”.

addNum is also wrapped in an extern directive and preceded by the EXPORT to ensure its visible outside of the shared library.

addNum2 is similarly wrapped by extern and EXPORT but calls a function that is invisible in the shared library called notExported. I only did this to show it is possible.

If you examine the library with nm -gU you will see that notExported is in fact not exported which means we aren’t polluting the namespace for our clients.

nm -gU libMacExampleAddin.dylib
00000f40 T _addNum
00000f70 T _addNum2

The nm options are:
-g     Display only global (external) symbols.
-U     Don’t display undefined symbols.

The Excel Spreadsheet using the Addin – MacExampleExcel

Before you start Excel copy MacExampleAddin/Debug/libMacExampleAddin.dylib to the MacExampleExcel directory. When you start Excel you’ll need to accept macros, but don’t worry there’s nothing bad in there, just some VBA to access the shared library.

addin-test.xlsm

When you start Excel it will ask you to accept macros (say yes) but it may not start the VBA editor so select menu Tools -> Macro -> Visual Basic Editor (I’ve exported the basic module here).

As you can see there are two sorts of functions, Private and non-private functions. The private ones are not visible to Excel but the rest are. The reason for separation is because that’s the only way I could get it to work, but conveniently the shared library functions are wrapped and hidden in the private functions and referenced by Lib “….”. The public functions simply call the private ones to return the values you see in the spreadsheet.

As you’ll see I’ve shown functions with absolute paths (addFunctionAbsolute) which only work for me. Yes, not good but it shows it works and you can change it if you want to make it work for your Mac. Secondly there are examples that assume a library is present in the same directory as the Excel spread sheet (addFunctionLocal). These also work if you look at the spreadsheet. Finally there are relative paths (addFunctionRelative) which don’t work but should, but don’t know why.

But what you should see is this:

cdt-excel

 

The C Main program that uses the Addin – MacExampleMain

This is a simple  main program again generated by CDT but as a “Hello World C++ Project” with MacOSX GCC tool chains. I shan’t detail all the changes as you can look at them yourself but basically they are:

  • added -arch i386 to all the tools
  • added the include path to MacExampleAddin/src
  • added the library path to MacExampleAddin/Debug
  • added the library MacExampleAddin (without the preceding lib and .dylib extension)
  • used clang++ and C++11
  • used clang and C11
  • changed main.cpp to test addNum as follows:
if (addNum(1,2) != 3) {
  cerr << "FAIL: Wrong number from addnum expected 3 and got " << addNum(1,2) << endl;
}

For your convenience there is an Eclipse launch that sets up DYLD_LIBRARY_PATH path to allow you to run the project in Eclipse as follows:

Run -> Run History -> MacExampleMain

You should see this in the console:

PASS: Correct number from addnum expected 3 and got 3

You need to set DYLD_LIBRARY_PATH to point to the directory where the dylib is if you intend to run the program in a terminal window:

peter$ ./Debug/MacExampleMain 
dyld: Library not loaded: libMacExampleAddin.dylib
  Referenced from: /Users/peter/Documents/eclipse/workspace-cdt/MacCPPExcelAddinExample/main/./Debug/MacExampleMain
  Reason: image not found
Trace/BPT trap: 5

Fails because the library can’t be found.

peter$ export DYLD_LIBRARY_PATH=../addin/Debug/
peter$ ./Debug/MacExampleMain 
PASS: Correct number from addnum expected 3 and got 3

This works because the library can be found.

But you don’t need to set DYLD_LIBRARY_PATH  if the library is in the current working directory:

peter$ ls -l
-rwxr-xr-x  1 peter  staff  17312 30 Jul 21:04 MacExampleMain
-rwxr-xr-x  1 peter  staff   4704 30 Jul 21:04 libMacExampleAddin.dylib

peter$ ./MacExampleMain 
PASS: Correct number from addnum expected 3 and got 3

 

© Copyright 2015, Peter Lappo, SMR Ltd.

Categories
Technology

Git Branching

            I like to use are two kinds of branches, features (aka enhancement / story) and issues. A single feature can encapsulate a group of enhancements or even a sprint. We can be flexible here as creating too many branches will slow us down. An issue is a bug, a configuration change or some kind of task that involves a code change.

The naming convention is:-
feature-name e.g. feature-add-currency-prices
issue-name e.g. issue-prices-not-appearing

Seeing which Branches are Available

git branch
Or for remote branches not in your repository
git branch -a

Creating a Branch

To create a branch on an project that you have previously cloned and as a side effect also switch to the branch
git checkout -b feature-add-currency-prices
Make some changes, commit them in the usual way. Then push to the repository and add the branch to the repository so that others can see it
git push -u origin feature-add-currency-prices
And simply do
git push
subsequently

Using an Existing Branch

To switch to an existing branch first make sure you have the latest repo which includes the branch
git pull
Then checkout the branch and switch to it
git checkout feature-add-currency-prices
Make git changes and commit in the usual way and then when its time to push
git push -u origin feature-add-currency-prices
After that git remembers your branch so you can simply type
git push

Merging to Master

Finally merge to master
git checkout master
git pull
git pull origin feature-add-currency-prices
git push origin master

Merging changes from Master to a branch

git checkout master
git pull
git checkout feature-add-currency-prices
git pull origin feature-add-currency-prices
git merge master
-- there maybe conflicts at this point
git status
-- after all conflicts are resolved
git push origin feature-add-currency-prices

Issues

Eclipse didn’t do a good job of branching but once the branch is setup it does recognise the branch and can be used for commit and after the first manual push.

Tags

To tag the master after a successful test to say v-0.2.6

git tag -a v-0.2.6 -m 'v-0.2.6'
git push origin v-0.2.6

Tips

To see what commits are waiting to be pushed
git log --branches --not --remotes
For a list of files to be pushed, run
git diff --stat --cached origin/master
To delete a branch
git branch -d branch_name
To remove from origin
git push origin :branch_name

References

https://www.atlassian.com/git/workflows#!workflow-feature-branch
http://reinh.com/blog/2009/03/02/a-git-workflow-for-agile-teams.html
http://marklodato.github.io/visual-git-guide/index-en.html
http://nvie.com/posts/a-successful-git-branching-model
http://rogerdudler.github.io/git-guide/

Categories
NoSQL Technology

Cassandra Data Modelling Videos

            Watch this first...<a href="http://www.slideshare.net/patrickmcfadin/the-data-model-is-dead-long-live-the-data-model">The data model is dead long live the data model</a> and <a href="https://www.youtube.com/watch?v=px6U2n74q3g"> video</a>

And then….The worlds next top data model and video

Finally…Become a super modeler and video

Other modelling videos including the three above

Categories
Data Gathering

Financial Market Data Websites

            <a href="http://www.kibot.com">www.kibot.com</a> - bid/ask, some tick data, no volumes for FX

www.tickdatamarket.com/forex.php – pricy €280 per currency pair

www.quandl.com/collections – general economic data, some stocks, daily at best

datacatalog.worldbank.org and data.worldbank.org – world bank open data

www.fxdd.com/us/en/forex-resources/forex-trading-tools/metatrader-1-minute-data – meta trader format

http://www.histdata.com/download-free-forex-data/?/ascii/1-minute-bar-quotes – CSV

www.trading-tools.com/index.htm – data downloader, eod only, $59+

www.stockhistoricaldata.com/daily-download – either manual or via xls plugin, takes eod from yahoo

eoddata.com/products/default.aspx – free eod 1 min ohlc, but only 30 days, other not expensive

www.csidata.com – source for yahoo it seems – no fx

quantquote.com/historical-stock-data – stocks only

www.netfonds.no/quotes/paperdump.php?paper=EURUSD.FXSB – free eod only

knoema.com/atlas – all sorts of data including some economics

www.dukascopy.com/swiss/english/marketwatch/currency_index – 1 min fx ohlcv

stooq.com/db/h – 5 min fx no vol data also equities

www.fxhistoricaldata.com/EURUSD – hourly data fx, no volume, free but limited pairs

www.histdata.com/download-free-forex-data/?/ascii/1-minute-bar-quotes – lots of zip files but no volume

Categories
Gradle Technology

Maven Central Gradle Guide

Sign up to sonatype first and checkout this guide. Copy suitable bits from this working gradle build file and see LogOnce.

Will also need a PGP key, install GPG and GPG user guide and sonatype PGP help is here.