Tuesday 9 September 2014

Bluetooth on Virtual box with Mac OS x 10.9.4 as host

You want to have bluetooth on your Linux/Window in VirtualBox while using Mac OS X as host. The bug (Ticket #2372) has been filed six (6!!!) years ago, go figure.

For 10.9.4 on MBP 2014 you need to execute following:
$sudo su
$launchctl unload /System/Library/LaunchDaemons/com.apple.blued.plist
$kextunload -b com.apple.iokit.IOBluetoothSerialManager
$kextunload -b com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport

Note that the actual extension may vary depending on your Mac hardware, so you can simple list modules and try to unload each and one of them until you can connect bluetooth on virtual box.

$kextstat | grep -i bluetooth

Cheers

Wednesday 27 August 2014

Developing for Nordic Semiconductor nRF51822


What 

This post describes installation and first simple example running dev kit nRF51822 on Mac OS X 10.9.4.

Why

Its either you who will rule Internet of Things, or Internet of Things will rule you. Pick a side and spot wining.
I got my hands on nRF51822 and since I have switched to Mac, the installation instructions for mac are confusing... there was no obvious way to run it from thus this is a mental note to my self how to set up environment. So here you go, after tutorial you will be up and running in no time :) with multiple options.


How

First install needed shit, its all below. The instructions are minimal and assumes you can figure out some simple stuff. If not ask, Google, change carrier.

1. Eclipse

I have many copies of eclipse for different developing purpose containing different language. The reason is that plugins do brake each other and I rather reinstall single plugin then have all my "life"dependent on one eclipse.
  1. Download Eclipse http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers/lunar (I used eclipse-cpp-luna-R-macosx-cocoa-x86_64.tar.gz)
  2. Extract it
  3. Rename it to eclipseARM
  4. Enter folder and rename eclipse.app to ArmLipse.app
  5. Move folder to /Applications/ or /User/Applications/ 

1.1 Plugins

  1. Lauch ArmLipse
  2. Create new workspace nRF51 (keep it clean)
  3. Help->Install new software
  4. Add http://embsysregview.sourceforge.net/update 
  5. and install embsysregview
  6. Go back, select all available sites, search and install GDB Hardware Debugging
  7. Go back and add site http://gnuarmeclipse.sourceforge.net/updates and install plug in for arm
  8. Go back and add http://nrf51osx.sourceforge.net/updates and install nordic plugin

2.  GNU Tools

  • Get tar for mac from here https://launchpad.net/gcc-arm-embedded/ (I got gcc-arm-none-eabi-4_8-2014q2)
  • Extract it.

3. Jlink

  1. Get libusb to start with brew install libusb
  2. Get Jlink from http://www.segger.com/jlink-software.html
  3. You will need to enter number from the biggest chips on the board
  4. I got JLink_MacOSX_V490b.pkg, double tap and follow instructions

4. Libraries

  1. You will need firmware either s110, s120, s130 depending on the purpose. You can get then form nordic Semiconductor site.
  2. Create folder development in /User/your_name/ and create subfolder nrf51
  3. Copy, SDK, Firmware and GNU tools to it
  4. Mine looks: 
    ->tree -CL 1 nrf51
    nrf51
    ├── gcc-arm-none-eabi-4_8-2014q2
    ├── nrf51_sdk_v6_0_0_43681
    ├── s110_nrf51822_7.0.0
    ├── s120_nrf51822_1.0.0
    └── s130_nrf51822_0.5.0-1.alpha
    
    5 directories, 0 files
    

5. Configure

5.1 Eclipse - hardway

In case you are insane and have no life play here:
  1. embsysregview: Copy sdks/nrf51/nrf51_sdk_v6_0_0_43681/nrf51822/SVD/nrf51.xml to Applications/eclipseARM/plugins/org.eclipse.cdt.embsysregview.data_0.2.4.r168/data/cortex-m0/Nordic
    cd ~
    mkdir Applications/eclipseARM/plugins/org.eclipse.cdt.embsysregview.data_0.2.4.r168/data/cortex-m0/Nordic
    cp development/sdks/nrf51/nrf51_sdk_v6_0_0_43681/nrf51822/SVD/nrf51.xml Applications/eclipseARM/plugins/org.eclipse.cdt.embsysregview.data_0.2.4.r168/data/cortex-m0/Nordic/ 
    
  2. Configure Environment, add this to the .bashrc (!!!change username to your username!!!)
    cd ~
    export GNU_INSTALL_ROOT=/Users/username/development/sdks/nrf51/gcc-arm-none-eabi-4_8-2014q2
    export GNU_VERSION=4.8.4
    
  3. Now, lets remove Nordics bug from Makefile.posix, just delete these lines (why are they there???)
    GNU_INSTALL_ROOT := /usr/local/gcc-arm-none-eabi-4_8-2014q1
    GNU_VERSION := 4.8.3
    
  4. Now follow the rest of configuration described in n29 v1.0 from 1.2 and make sure you can compile blinky.
  5. Dont rewrite, c-c c-v, into startup window
    mon speed 10000
    mon endian little
    mon flash download = 1 
    mon flash device = NRF51822
    mon reset 0

5.2 Eclipse easy way

If you want time over for your girlfriend just follow these steps:
  1. Create a project how its is described in http://sourceforge.net/p/nrf51osx/wiki/create%20a%20project/
  2. You done!

6 Make that F*** blink

So all done and you happy, almost. Go get this awesome programmer http://sourceforge.net/projects/rknrfgo/. Install, fire up terminal and run:

cdccontrol disable


Then fire up rknrfgo (fancy name) and program your device :) 

6.1 Bug fixing

Everybody is changing everything and fixing bugs all over the places. Which leads that there is HUGE amount of inconsistency in EVERY single post you will read or software you install. Trust no one.
  1. When you created project and if you are using SDK 6+ (and you should) uploading ldscripts/mem.ld will be faulty. To make it work just c-c c-v following (assuming you select softdevice :))
    /* 
    ** Linker script to configure memory regions. 
    **
    ** Device type: xxaa, 256K flash, 16K RAM
    ** Device type: xxab, 128K flash, 16K RAM
    **
    ** Softdevice:  none, all memory available to device,  Code start: 0x0000 , xxaa:0x40000, xxbb:0x20000, RAM start: 0x20000000 
    ** Softdevice:  s110, uses 80K code, 8K RAM,   Code start: 0x16000, xxaa:0x2c000, xxbb:0x0c000, RAM start: 0x20002000 
    ** Softdevice:  s120, uses 96K code, 10K RAM,   Code start: 0x18000, xxaa:0x28000, xxbb:0x08000, RAM start: 0x20002800 
    */ 
    
    GROUP(-lgcc -lc -lnosys)
    
    MEMORY
    {
      FLASH (rx) : ORIGIN = 0x00016000, LENGTH = 0x2C000
      RAM (rwx)  : ORIGIN = 0x20002000, LENGTH = 0x2000 
    }
    
    /*
    ** All permutations, copy one into the MEMORY section above, removing the leading '**'s
    **
    ** xxaa, no softdevice
    **  FLASH (rx) : ORIGIN = 0x00000000, LENGTH = 0x40000  
    **  RAM (rwx)  : ORIGIN = 0x20000000, LENGTH = 0x4000 
    **
    ** xxab, no softdevice
    **  FLASH (rx) : ORIGIN = 0x00000000, LENGTH = 0x20000 
    **  RAM (rwx)  : ORIGIN = 0x20000000, LENGTH = 0x4000 
    **
    ** xxaa s110
    **  FLASH (rx) : ORIGIN = 0x00016000, LENGTH = 0x2C000
    **  RAM (rwx)  : ORIGIN = 0x20002000, LENGTH = 0x2000 
    **
    ** xxab s110
    **  FLASH (rx) : ORIGIN = 0x00016000, LENGTH = 0x0C000
    **  RAM (rwx)  : ORIGIN = 0x20002000, LENGTH = 0x2000 
    **
    ** xxaa s120
    **  FLASH (rx) : ORIGIN = 0x00018000, LENGTH = 0x28000
    **  RAM (rwx)  : ORIGIN = 0x20002800, LENGTH = 0x1800 
    **
    ** xxab s120
    **  FLASH (rx) : ORIGIN = 0x00018000, LENGTH = 0x08000
    **  RAM (rwx)  : ORIGIN = 0x20002800, LENGTH = 0x1800 
    **
    */
    

7 What you get?! This beauty! :)

Dat Feeling of the #blink! #nordic #ble #internetofthings #iot #sanfrancisco #arduino #california #oneplus #summershots #SoC #nrf51822

Acknowledgment

The biggest thanks goes to the people who spread they knowledge all over the internet, and did load of coding. Go check them out, donate, send flower or fruits!

Roland King (King indeed)
http://sourceforge.net/projects/rknrfgo/
http://sourceforge.net/projects/nrf51osx/

Other sources of ideas
http://hg.cmason.com/nrf
http://robolabwiki.sdu.dk/mediawiki/index.php/Nordic_Semiconductor_nRF51822_development
http://erlblog.lewin.nu/2013/12/setting-up-eclipse-under-mac-os-x-for.html

Tuesday 8 April 2014

Recovering a corrupt Eclipse workspace

I had a computer crash with a running Eclipse which resulted in a corrupt workspace. I was simply getting message that workspace is in use. While there is many proposed solutions on the internet the simplest one is to open .metadata folder in your workspace, make sure that you can see invisible (hidden) files there and remove .lock file. Works like a charm and preserves all projects. :)

Wednesday 19 March 2014

Customizing django-registration


Whatz up?!

The integration of django-register is awesome! But if you want to customize it with additional models it can be confusing.

Here in example I will show how to create your custom backend, form to handle it (custom validator and model choice field).

Note, the django I use is version v1.5.4 django-registration v1.0. Some line breaks are imposed due to layout of this page.

Thus, bellow you will find everything you need to:
  • Create custom backend
  • Add custom form 
  • Create custom validation
  • Badabung! :D

Below is and example extending
from registration.backends.default.views.RegistrationView
First of all, inherit RegistrationView, then override register() and get_form_class() functions. In the examples to follow it is assumed that the class is saved to regbackend.py.
#regbackend.py
from registration.backends.default.views import RegistrationView

#create custom registration backend
class StudentBackend(RegistrationView): 
    '''
    we just want to override register and fix profile creation
    ''' 
    def register(self,request, **kwargs):
        from usermanager.models import Profile
        user=super(StudentBackend,self).register(request, **kwargs)
        user.first_name = kwargs['first_name']
        user.last_name = kwargs['last_name']
        user.save()
        profile = Profile.objects.create(
                                user=user, course=kwargs['course'])
        profile.save()
        
    def get_form_class(self, request):
        """
        Return the default form class used for user registration.

        """
        from usermanager.form import UserRegistrationForm
        return UserRegistrationForm    
Below is the code from urls.py. Nothing fancy, just import your registration backend, and form, and redirect accounts/register/ to your backend.
#urls.py
from regbackend import StudentBackend
from form import UserRegistrationForm

urlpatterns = patterns('',
    url(r'^accounts/register/$',
        StudentBackend.as_view(form_class = UserRegistrationForm),
        name='registration_register'),
    (r'^accounts/', include('registration.backends.default.urls')),
    #more url configs
)
Following code is from form.py:
#form.py
from django import forms
from django.contrib.auth.models import User
from usermanager.models import Course, Profile
from registration.forms import RegistrationForm
from django.utils.translation import ugettext_lazy as _



attrs_dict = { 'class': 'required' }

#this function checks for the allowed domain:
def affiliation_valid_email(value):
    allowed_domain=['domain.se','domain.se']
    #not the prettiest soliution 
    v_error=_("Email does not match allowed domains (%s, %s)"
                           %(allowed_domain[0],allowed_domain[1])
    if value.split('@')[1] not in allowed_domain:
        raise forms.ValidationError(v_error)
    else:
        return value


#the registration form is inherited from django-registration   
class UserRegistrationForm(RegistrationForm):  
    
    class Meta:
        model = User
        fields = ("first_name",
                  "last_name",
                  "username", 
                  "email", 
                  "password1", 
                  "password2")
    #username is enforced to some particular rules 
    u_error = _("User name needs to match domain convention.")   
    username = forms.RegexField(regex=r'^\w',
                                max_length=9,
                                widget=forms.TextInput(attrs=attrs_dict),
                                label=_("Username - same as in domain!"),
                                error_messages={ 
                                'invalid': u_error })
    first_name = forms.CharField(max_length=30)
    last_name = forms.CharField(max_length=30)
    course = forms.ModelChoiceField(queryset=Course.objects,required=True)
    email = forms.EmailField(validators=[affiliation_valid_email],
                             widget=forms.TextInput(
                                     attrs=dict(attrs_dict,maxlength=75)
                                     ),
                                     label=_("domain - Email address"))
    

    
    def clean(self):
        #clean with super
        ue_error = _("Email and user name do not match.")
        super(UserRegistrationForm, self).clean()
        #this is just checking that username is the same as in email
        if self.cleaned_data['email'].split('@')[0] == 
                                        self.cleaned_data.get("username"):
            return self.cleaned_data
        else:
            raise forms.ValidationError(ue_error)
         
    def save(self, commit=True):
        user = super(UserRegistrationForm, self).save(commit=True)    
        user.email = self.cleaned_data["email"]
        user.first_name =  self.cleaned_data["first_name"]
        user.last_name =self.cleaned_data["last_name"]
        profile = Profile(
                   systemuser=user, course = self.cleaned_data["course"])

    if self.commit:
        user.save()
        profile.save()
    return user

Models used in this example:
#models.py

class Course(models.Model):
    code = models.CharField(max_length=6)
    teacher = models.CharField(max_length=30)
    
    class Meta:
        app_label = "usermanager"
        verbose_name = "Course"
        verbose_name_plural = "Courses"
    
    def __unicode__(self):
        return "%s" % self.code
    
class Profile(models.Model):
    user=models.ForeignKey(User, unique=True)
    course=models.ForeignKey('usermanager.Course')
    created = models.DateTimeField(auto_now_add=True)
    updated_local = models.DateTimeField(blank=True, null=True)
    

    def getCourse(self):
        return "%s" %self.course
    
    def save(self,*args,**kwargs):
        self.updated_local = datetime.now()
        super(Profile, self).save(*args,**kwargs)
    
    def __unicode__(self):
        return "Profile: %s" %self.user

Finally, remember to enable registration in installed apps before running
python manage.py syncdb
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'django.contrib.admindocs',
    'registration',
)
That's it foks, happy coding! L

Monday 22 July 2013

Quick and dirty multiply raw file conversion

Multi-raw2dng-converter

I asume that you know all about installiation and usage of Magic Lanters raw video functionality. 

If you like me find your self with many raw files, using raw2dng manually can be pain in a**.  Here you have python 3.x script working and tested on Windows 8. Given the argument -d "file directory" it will look for any subfolders there and for each raw file in subfolder will create new folder and will run raw2dng on that file. The resulting dng will end up in folder named "ra_file_name" without RAW. The script utilizes as many new processes as there cores in CPU.

Note that this script is just quick and  dirty way to fix my problem. I hope it will be useful for other as well. Feel free to modify and extend.

Usage:
D:\dev\python multiConverter.py -d "F:\Images\2013\0722"

Output:
Successfully processed: F:\Images\2013\0722\1\M17-2243.RAW

Script:
#!/usr/bin/python
#searches for raw files and converts then into a directory with the 
#same name as file
import glob
import sys, getopt
import os, multiprocessing, queue, subprocess
from threading import Thread, Lock

FILE = 0
DIR = 1
RAW2DNG = r"C:path\to\raw2dng.exe"
def optionsGet(argv):
 inputdir = ""
 try:
  opts, args = getopt.getopt(argv,"hd:",["ddir="])
 except getopt.GetoptError:
  print('multiconverter.py -d ')
  sys.exit(2)
 for opt, arg in opts:
  if opt == "-h":
   print("multiconverter.py -d ")
   sys.exit()
  elif opt in ("-d", "--ddirectory"):
   inputdir = arg
 return inputdir

def getCPUs():
 # Windows
    try:
        res = int(os.environ['NUMBER_OF_PROCESSORS'])
        if res > 0:
            return res
    except (KeyError, ValueError):
        pass

def createFolder(name):
 pass
 

def getFileNames(name):
 return glob.glob(name + "\*.RAW")

def getsubdirs(inputdir):
 file_list = []
 dirs=os.listdir(r'%s' %inputdir)
 for d in dirs:
  tmp = r'%s\%s' %(inputdir,d)
  file_list.append(tmp)
 return file_list
  
def createJobs(dir_list, q):
 for d in dir_list:
  files = getFileNames(d)
  for f in files:
   q.put((f,d))
   
  
   
def deployWorkers(file_list, l, num_workers, func):
 jobs = multiprocessing.Queue()
 createJobs(file_list, jobs)
 workers = []
 for i in range(num_workers):
  jobs.put(None)
  tmp = multiprocessing.Process(target=func, args=(jobs,))
  tmp.start()
  workers.append(tmp)
 for worker in workers:
  worker.join()
  
def raw2dngWorker(queue):
 while True:
  job = queue.get()
  if job == None:
   break
  else:
   #enter right dir
   os.chdir(job[DIR])
   #create folder
   folder = job[FILE].split(".")[0]
   os.makedirs(folder)
   #change dir
   os.chdir(folder)
   proc = subprocess.Popen([RAW2DNG, job[FILE]] , 
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT, )
   stdout_value, stderr_value = proc.communicate()
   if not stderr_value:
    print("Successfully processed: %s" %job[FILE])
   else:
    print("ERROR %s " %stderr_value)
   
   
 
   
if __name__ == "__main__":
 inputdir = optionsGet(sys.argv[1:])
 cores = getCPUs()
 l = Lock()
 file_list = getsubdirs(inputdir)

 #start cores = workes
 deployWorkers(file_list, l, cores, raw2dngWorker) 


Cheers

Friday 24 August 2012

Extending NS3 with your module and extra libraries

I have been working on custom module in ns3 for some time now. Due to many changes in NS3, like bug fixes, extensions, improvements, I decided to move to latest version.

This decision consumed couple of days and several gray hair, due to lack of documentation. The biggest difference for custom module developer is the new waf build system and thus how build works.

Reading http://www.nsnam.org/wiki/index.php/HOWTO_use_ns-3_with_other_libraries it seems very simple to add additional libraries, however the tutorial is not valid for module. Because the suggested build steps will only link the individual files with external libraries and NOT the module. The flags are only passed when compiling indivirual object, the bug (?) is reported.

So here is how I managed to solve it

First, add configuration in your wafscript script
def configure(conf):
     conf.env['xercersc'] = conf.check(mandatory=True, lib='xerces-c', uselib_store='XERCES')
     conf.env['ldl'] = conf.check(mandatory=True, lib='dl', uselib_store='LDL')
 
make sure you change naming to your libs, note also the name of library, you have to omit "lib", "l" and only go by the name. Having mandatory=True will abort the compilation if waf can not find the library.

Second and last

def build(bld):
    #......
    #...... code omited
    #......
    module.use.append("XERCES")
    module.use.append("LDL")
    #......
    #...... the rest of build code
Delete build directory run
./waf configure && ./waf build
If you still having problems with linking it is very useful to run
./waf --no-task-lines -v
In that case after each build step waf will output runner were the compiler and all passed flags are visible. From there you can continue bug hunt :)

Cheers

Friday 10 August 2012

TinyOS on Ubuntu 12.04 update error, the fix

 TinyOS released the update to tinysos-tools package. However, if updating from GUI of update manager you will get an error

The action would require the installation of packages from not authenticated sources.

And update will be aborted. To fix it, simply open terminal and run
sudo apt-get update
sudo apt-get upgrade
You will get two warning, proceed by answering Y and you are done!