THE BLOG

13
May

Speed up search in Visual Studio Code by excluding folders

Add the following to your Workspace or User settings:

"search.exclude": {
    "**/.git": true,
    "**/node_modules": true
}
27
Apr

Permenently allowing an insecure registry on docker-machine / boot2docker

Adding the --insecure-registry flag has been a bit of a pain for me. I’d ssh into the boot2docker machine and edit the file before restarting docker on the VM. Unfortunately every time the docker machine restarted I would lose the configuration and have to perform the same steps again.

Fortunately, I discovered the configuration files for the docker-machines and that it allows you to specify insecure registries by configuration.

Do the following to enable your insecure registry once and for all:

Edit the configuration file for your docker-machine (default in this case):

vi ~/.docker/machine/machines/default/config.json

Locate the `InsecureRegistry` key and simply append your insecure registry to the array.

    "HostOptions": {
        "Driver": "",
        "Memory": 0,
        "Disk": 0,
        "EngineOptions": {
        "ArbitraryFlags": [],
        "Dns": null,
        "GraphDir": "",
        "Env": [],
        "Ipv6": false,
        "InsecureRegistry": [
            "insecure-registry.mydomain.com"
        ],

Then re-provision the machine with:

docker-machine provision default

Note: re-provisioning will likely lose any manual changes you made to the VM.

06
Apr

Excluding node_modules from Sublime Text Ctrl+P file search

Add the following to user preferences (Preferences / Settings – User):

"folder_exclude_patterns": [".git", "node_modules", "bower_components"]

Make sure to restart sublime for the changes to take effect.

The problem with this setting is that it also removes these folders from the sidebar.

If you still want to see the files in the sidebar but still exclude them from search and Ctrl + P you can instead use binary_file_patterns:

"binary_file_patterns": [".git/", "node_modules/", "bower_components/"]

Note that a forward slash is required at the end for directories.

Remember to restart Sublime again.

29
Mar

Running Docker without sudo

After installing docker on linux you might find that you have to run all docker commands with sudo. It is possible to add your username to the docker group so that you can run Docker without using sudo:

sudo usermod -a -G docker ${USER}

You may need to log out and back in for it to take effect.

24
Mar

Adding gzip compression to a Rancher HAProxy Load Balancer

Adding gzip compression to a Rancher HAProxy Load Balancer

Say you want to gzip application/json coming from your application you can put a Rancher Load Balancer (HAProxy) in front and add the following config to the defaults section:

compression algo gzip
compression type application/json

This will then gzip all responses with Content-Type: application/json if the client supports compression.

A simple benchmark suggests that doing it at the LB instead of within the application is more performant than, say, using the compression middleware in a node.js express application

For a more complete set of MIME types which can be compressed, use this:

compression type text/html "text/html; charset=utf-8" text/html;charset=utf-8 text/plain text/css text/javascript application/x-javascript application/javascript application/ecmascript application/rss+xml application/atomsvc+xml application/atom+xml application/atom+xml;type=entry application/atom+xml;type=feed application/cmisquery+xml application/cmisallowableactions+xml application/cmisatom+xml application/cmistree+xml application/cmisacl+xml application/msword application/vnd.ms-excel application/vnd.ms-powerpoint image/svg+xml
11
Apr

Deleting all files which do not contain @2x in Windows

[codesyntax lang="bash"]
for /f "eol=: delims=" %F in ('dir /b /a-d * ^| find /v /i "@2x"') do del "%F"
[/codesyntax] From http://stackoverflow.com/questions/10788075/dos-batch-for-loop-to-delete-files-not-containing-a-string
19
Mar

Analysing the mySQL Slow Query log file

mysqldumpslow -s c -t 10 -r /var/log/mysql/mysql-slow.log
18
Mar

Custom sorting order in LINQ (ORDER BY WEIGHTING)

I have developed a C# LINQ extension method to allow very flexible ordering by assigning weightings to the sort keys of the elements to be ordered. I was inspired by the functionality offered by SQL Server’s ORDER BY CASE WHEN:

[codesyntax lang=”sql”]

ORDER BY
CASE SEASON
WHEN 'WINTER' THEN 1
WHEN 'SPRING' THEN 2
WHEN 'SUMMER' THEN 3
WHEN 'AUTUMN' THEN 4
END

[/codesyntax]

The extension method I have created lets you pass a lambda function which allows the use of logic to apply custom weightings to the sort keys.

[codesyntax lang="csharp"]
var data = dt.Select(g => new
{
    Season = g.season,
    AverageTemp = g.temp
}).OrderByWeight(a => a.Season, x =>
{
    if (x == "WINTER") return 1;
    if (x == "SPRING") return 2;
    if (x == "SUMMER") return 3;
    if (x == "AUTUMN") return 4;
    return 99;
});
[/codesyntax]

The above will return an IOrderedEnumerable sorted by Season in the order WINTER, SPRING, SUMMER, AUTUMN. However, the OrderByWeight extension method provides even more powerful functionality as the lambda allows us to perform string comparisons and more:

[codesyntax lang="csharp"]
var data = dt.Select(g => new
{
    Year = g.year,
}).OrderByWeight(a => a.Year, x =>
{
    if (x.Contains("2008")) return 1;
    if (x.Contains("2009")) return 2;
    if (x.Contains("2010") return 3;
    return 99;
});
[/codesyntax]

Using the above, a non-trivial list such as {‘London 2010’, ‘Leeds 2008’, ‘Cardiff 2009’, ‘Glasgow 2011’} can be easily sorted to {‘Leeds 2008’, ‘Cardiff 2009’, ‘London 2010’, ‘Glasgow 2011’}

The extension method used to implement this functionality is:

[codesyntax lang="csharp"]
public static IOrderedEnumerable<TSource> OrderByWeight<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TKey, int> weighting) where TKey : IComparable
{
    Dictionary<TSource, int> order = new Dictionary<TSource, int>();
    foreach (TSource item in source)
    {
        if (!order.ContainsKey(item)) order.Add(item, weighting(keySelector(item)));
    }
    return source.OrderBy(s => order[s]);
}
[/codesyntax]

It has not been throughly unit tested yet but it should have the same behaviour as GroupBy. Whilst I have only provided the C# implementation I’m sure it can be converted to other languages using the CLR.

25
Feb

String.Capitalize in C# (PHP’s ucwords in C#)

I have been playing around with C# recently and found the need for functionality similar to that provided by PHP’s ucwords() function. I dont think this exists in C# as standard so here is a simple extension method to achieve ucwords in C# using a little bit of LINQ (the extremely useful Aggregate extension method) to achieve it:

public static string Capitalize(this String s)
{
  return s.ToCharArray().Aggregate(String.Empty,
    (working, next) =>
      working.Length == 0 && next != ' ' ? next.ToString().ToUpper() : (
        working.EndsWith(" ") ? working + next.ToString().ToUpper() :
          working + next.ToString()
    )
  );
}

With this included in your name space, the extension method can be used on any String instance like:

string myString = "this sentence needs capitalization!";
Console.WriteLine(myString.Capitalize());
//This Sentence Needs Capitalization!

Short but sweet!

26
Aug

Cloning Zend_Config without side-effects

Cloning in PHP5

PHP5 with its much improved object-oriented style passes objects by reference by default. To create a copy of an object, a new construct ‘clone’ is introduced. Cloning an object in PHP5 creates a shallow copy – any properties that are references to other variables, will remain references. Additionally, PHP5 brings the __clone() magic method to allow you to customise the cloning behaviour of objects, making those referenced objects full copies if that is the desired behaviour.

Zend_Config

Zend_Config is a component of the Zend Framework. Zend_Config makes it trivial to load configuration data into PHP from a range of file types.

Using Zend_Config is easy:

[codesyntax lang="php"]
$config = new Zend_Config_Xml('/path/to/config');
echo $config->database->host;
[/codesyntax]

In a particular scenario of mine I have a heirachy of config files, each inheriting data from it’s parent and overwriting keys if necessary. Merging two config files to get the resultant is also trivial:

[codesyntax lang="php"]
(void) $parent->merge($child);
[/codesyntax]

The Zend_Config instance $parent has now been augmented with the values of the Zend_Config instance $child and keys may have been overwritten in the parent if a value for that key existed in the child.

Here lies my problem: What I really wanted to do was keep $parent as it was whilst creating a new instance that was the product of $parent merged with $child.

Easy:

[codesyntax lang="php"]
$newConfig = clone $parent;
$newConfig->merge($child);
[/codesyntax]

Or not… Currently Zend_Config only implements a shallow clone which effectively means my $newConfig is still tied up with $parent. In fact if I make changes to $newConfig, those changes may well be reflected in $parent. This is not the side-effect-free behaviour I expected when using clone.

It is true that Zend_Config is typically meant to be used in read-only situations but the power of merging has been realised and implemented, making Zend_Config often used in read-write situations aswell (here writing refers to changing the values in the instance, not the source configuration file).

Here is an example of the problem:

[codesyntax lang="php"]
$parent = new Zend_Config(array('key' => array('nested' => 'parent')), true); //allow read-write for merging
$newConfig = clone $parent;
$newConfig->merge(new Zend_Config(array('key' => array('nested' => 'override')), true));
echo $newConfig->key->nested; // 'override'  - as expected
echo $parent->key->nested; // 'override' - I was expecting this to be 'parent'
[/codesyntax]

This is actually perfectly reasonable behaviour for a shallow clone – The nested data value was cloned only by reference so (Zend_Config) $newConfig->key is actually a reference to the original (Zend_Config) $parent->key object and so changing any one will affect the other. However, this is not what I would expect to happen when cloning a Zend_Config object.

Solution 1: toArray

The toArray solution is simple: Flatten the Zend_Config object to a simple array and create a brand new Zend_Config object with the data. All objects are copied into the array and we get a deep copy of the object.

[codesyntax lang="php"]
$parent = new Zend_Config(array('key' => array('nested' => 'parent')), true); //allow read-write for merging
$newConfig = new Zend_Config($parent->toArray(), true); //cast the parent object to an array and create a new Zend_Config
$newConfig->merge(new Zend_Config(array('key' => array('nested' => 'override')), true));
echo $newConfig->key->nested; // 'override'  - as expected
echo $parent->key->nested; // 'parent' - as expected
[/codesyntax]

Solution 2: Performing a Deep Clone

OK – That’s how deep cloning is achieved with the current release but what about the problem of the misleading and unexpected operation of the clone construct? The simple and intuitive solution is to create a __clone() method that generates a full deep clone which is completely independent:

[codesyntax lang="php"]
    /**
     * Perform a deep clone of this instance to allow side-effect free cloning.
     * @return void
     */
    public function __clone()
    {
        $data = array();
        foreach ($this->_data as $key => $value)
        {
            if ($value instanceof Zend_Config)
            {
                $data[$key] = clone $value;
            } else {
                $data[$key] = $value;
            }
        }
        $this->_data = $data;
    }
[/codesyntax]

Now we can do a full clone intuitively without having to cast to an array first:

[codesyntax lang="php"]
$parent = new Zend_Config(array('key' => array('nested' => 'parent')), true); //allow read-write for merging
$newConfig = clone $parent;
$newConfig->merge(new Zend_Config(array('key' => array('nested' => 'override')), true));
echo $newConfig->key->nested; // 'override'  - as expected
echo $parent->key->nested; // 'parent' - as expected
[/codesyntax]

Summary

To summarise, cloning a Zend_Config object doesn’t have intuitive results due to shallow cloning. To get a full clone we must cast the original object to an array before creating a brand new instance. Alternatively, the Zend_Config class should implement deep cloning to get the desired and expected behaviour.

Test cases, the extended class, and a patch can be downloaded here (tests work on R9566 as provided in the file).

Supporting Test Cases

[codesyntax lang="php"]
/**
 * This file is part of ZendConfigClone
 *
 * @package   ZendConfigClone
 * @author    Daniel Skinner
 */
require_once dirname(__FILE__) . '/../AbstractTest.php';
/**
 */
class ZendConfigCloneTest extends AbstractTest
{
    /**
     * 0 - Use original Zend_Config object R9566
     * 1 - Use new ZendConfigClone object
     *
     */
	const VERSION = 1;
    public function setUp ()
    {
        $this->baseConfig = self::factory(array(
            'value1' => 'base1',
            'value2' => array(
                'nestedkey' => 'nestedvalue'
            )
        ), true);
    }
    /**
     * As expected, flattening the object into an array prevents any
     * references from being maintained and allows for a complete clone
     * of the Zend_Config object.
     *
     */
    public function testToArrayDoesNotKeepReferences()
    {
    	$newConfig = self::factory($this->baseConfig->toArray(), true);
    	$newConfig->value1 = 'newvalue1';

    	$this->assertNotEquals($newConfig->value1, $this->baseConfig->value1);
    	$this->assertEquals($newConfig->value2->nestedkey, $this->baseConfig->value2->nestedkey);
    }
    /**
     * As expected, flattening the object into an array prevents any
     * references from being maintained and allows for a complete clone
     * of the Zend_Config object.
     *
     */
    public function testNestedToArrayDoesNotKeepReferences()
    {
        $newConfig = self::factory($this->baseConfig->toArray(), true);
        $newConfig->value2->nestedkey = 'newvalue1';

        $this->assertNotEquals($newConfig->value2->nestedkey, $this->baseConfig->value2->nestedkey);
    }
    /**
     * With the current Zend_Config, a shallow clone will copy
     * first level values correctly.
     *
     */
    public function testShallowCloneDoesNotKeepReferences()
    {
        $newConfig = clone $this->baseConfig;
        $newConfig->value1 = 'newvalue1';

        $this->assertNotEquals($newConfig->value1, $this->baseConfig->value1);
    }
    /**
     * Currently cloning Zend_Config isn't side-effect free.
     *
     * A shallow clone means deep nested Zend_Config objects are
     * not cloned and changing values in the cloned object still
     * affects the original.
     *
     * This test should pass with ZendConfigClone
     *
     */
    public function testDeepCloneDoesNotKeepReferences()
    {
        $newConfig = clone $this->baseConfig;
        $newConfig->value2->nestedkey = 'newvalue1';

        $this->assertNotEquals($newConfig->value2->nestedkey, $this->baseConfig->value2->nestedkey);
    }
    /**
     * Factory method for generating a concrete Zend_Config instance
     *
     * @param mixed $arg,... Arguments to pass to the constructor
     * @return Zend_Config
     */
    private static function factory()
    {
    	$args = func_get_args();
    	switch (self::VERSION)
    	{
    		//Zend_Config
    		case 0:
    			$refObj = new ReflectionClass('Zend_Config');
    			return $refObj->newInstanceArgs($args);
    			break;
    		//ZendConfigClone
    		case 1:
                $refObj = new ReflectionClass('ZendConfigClone');
                return $refObj->newInstanceArgs($args);
    			break;
    	}
    	throw new Exception('VERSION should be enum{0, 1}');
    }
}
[/codesyntax]

A new class with deep clone functionality for testing:

[codesyntax lang="php"]
/**
 * This file is part of ZendConfigClone
 *
 * @package   ZendConfigClone
 * @author    Daniel Skinner
 */
class ZendConfigClone extends Zend_Config
{
    /**
     * Perform a deep clone of this instance to allow side-effect free cloning.
     * @return void
     */
    public function __clone()
    {
        $data = array();
        foreach ($this->_data as $key => $value)
        {
            if ($value instanceof Zend_Config)
            {
                $data[$key] = clone $value;
            } else {
                $data[$key] = $value;
            }
        }
        $this->_data = $data;
    }
}
[/codesyntax]