Monday, August 04, 2014

Capacity calculation and planning using X-RAID2 with Dual redundancy

I have a RedayNAS Ultra 6 Plus that I initially configured with 6 2TB drives X-RAID2 and Dual redundancy. This gave me 8TB of space (which I never took the time to even try and understand why).

Recently I started getting SMART+ warnings about one of those drives and decided to replace it.
I remembered this animation of extending X-RAID2 and because My drive was already at 87% I decided to but not one but 2 new 3TB drives expecting my capacity to grow by and additional 1TB.

As it turns out this was not the case.

At first I thought there was an issue with the expansion process not being triggered triggering so I tried to check the check the box titled "Check and fix quotas on next boot. This process can take several minutes to more than an hour depending on disk capacity and the number of files on your volume." as was recommended somewhere but this also did not work.

finally reading through the forums I found the following explanation:

X-RAID2 dual-redundancy uses RAID-6 layers. RAID-6 layers require a minimum of four disks.


So how do you calculate capacity for a given set of hard drives configured for X-RAID2 DR (Dual redundency) ?

first of all - you MUST have 4 drives to use Dual redundancy.
now, look at the smallest drive and suppose it has size S1.
count how many drives have at least S1 (lets call this N1).
if N1>=4 then add to expected total capacity S1*(N1-2).
now repeat with "un-accounted-for space" on all drives.

e.g.
Suppose you have the following drives 1TB , 1TB , 2TB , 2TB, 3TB , 4TB
The smallest one would be 1TB and we have 6 drives with that "un-accounted-for" space on them,
so 1TB*(6-2) yields 4TB and leaves you with 0TB,0TB,1TB,1TB,2TB,3TB "un-accounted-for" space.
next we consider another 1TB layer on the last 4 drives  that yields 1TB*(4-2) = 2TB more and leaves you with 0TB,0TB,0TB,0TB,1TB,2TB "un-accounted-for" space.

So what if you now want to get more space? the first layer is "full" (i.e. it uses 1TB of all 6 drives) but the second layer can be expanded in several different ways with the minimal one being just expanding one of the first two drives by an additional 1TB  and thuse getting 1TB*(5-2) = 3TB for that layer.

This was about calculating... you should now be able to do some planning by your self.

As for me, after understanding the situation I am at 2TBx4+3TBx2 yielding  8TB I thing the future expansion path would be by first replacing two more of the 2TB drives to 4TB each, this will give me an additional 2TB (try to calculate it yourself - 2TB,2TB,3TB,3TB,4TB,4TB) - then continue like this to each time replace the small pair to become the new largest pair.

The advantage is that from now on I will only need to replace two HDs each time to get additional space.
The disadvantage is that I some of the additional space I pay for will only be used when I buy the next pair of HDs.

Tuesday, March 04, 2014

"TypeError: Object has no method 'match'" using angularJs

I was getting the following error:

TypeError: Object 4 has no method 'match'

In some angular code I have written.
took some time to debug but it turns out the isolated scope of one of my directives had a property that was not a string. somewhere inside angular it seems that it is expected that the properties of the isolated scope would all be strings and thus have a .match method....

so
   .directive('myDirective',
      function() {
return {
restrict: "E",
replace: false,
scope: {
        x:4,
},


resulted in the said error.

just so that you will know...

Thursday, January 02, 2014

Why was I getting a "java.lang.IllegalArgumentException: Type mismatch" when trying a projection query on GAE

I was about to ask the following Q on StackOverflow.com but ended up finding the answer myself before even posting so I decided to make this into a new blog post.

I have two code snippets which I am trying to run on the development server of Google App Engine SDK v1.8.8 for Java.

The first:

    return datastore.get(KeyFactory.createKey(Kinds.Provider,provider_id));
    
works as expected and returns the Provider Entity from the Data Store.

But when I tried to change it to a projection Query so that it will return only some of the fields like this: 

    Filter filter=new FilterPredicate(Entity.KEY_RESERVED_PROPERTY,
                                      Query.FilterOperator.EQUAL,
                                      KeyFactory.createKey(Kinds.Provider,provider_id));
    Query q = new Query(Kinds.Provider)
                .setFilter(filter)
.addProjection(new PropertyProjection("address", String.class))
       .addProjection(new PropertyProjection("last_modified", String.class));;
}
    PreparedQuery pq = datastore.prepare(q);
    log.info("query:" + q.toString());
    
    Entity result = pq.asSingleEntity();

I am getting a the following Exception:


    java.lang.IllegalArgumentException: Type mismatch.
    at com.google.appengine.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:96)
    at com.google.appengine.api.datastore.RawValue.asType(RawValue.java:61)
    at com.google.appengine.api.datastore.PropertyProjection.getValue(PropertyProjection.java:65)
    at com.google.appengine.api.datastore.EntityTranslator.createFromPb(EntityTranslator.java:29)
    at com.google.appengine.api.datastore.QueryResultsSourceImpl.processQueryResult(QueryResultsSourceImpl.java:199)
    at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:106)
    at com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureLoaded(QueryResultIteratorImpl.java:155)
    at com.google.appengine.api.datastore.QueryResultIteratorImpl.nextList(QueryResultIteratorImpl.java:110)
    at com.google.appengine.api.datastore.LazyList.forceResolveToIndex(LazyList.java:93)
    at com.google.appengine.api.datastore.LazyList.resolveToIndex(LazyList.java:73)
    at com.google.appengine.api.datastore.LazyList.resolveToIndex(LazyList.java:56)
    at com.google.appengine.api.datastore.LazyList.isEmpty(LazyList.java:260)
    at com.google.appengine.api.datastore.PreparedQueryImpl.asSingleEntity(PreparedQueryImpl.java:74)

as I was copying This exception For my StackOverflow.com Q I discovered that there was another exception before that

12:09:18 PM com.google.appengine.api.datastore.dev.LocalCompositeIndexManager updateIndexFile
SEVERE: Unable to write some_path\war\WEB-INF\datastore-indexes.xml
java.io.FileNotFoundException: some_path\war\WEB-INF\appengine-generated\datastore-indexes-auto.xml (Access is denied)

Now it makes more sense...
Projection Queries require indexes to be created as explained here If the datastore-indexes-auto.xml is read only... well you now know what happens then...

well...
I thought this was the end of it but no... After making the file writeable, GAE indeed added an index to it but I still get the original 
IllegalArgumentException 

logging the querys .toString() yeilded:

SELECT last_modified, address FROM Provider WHERE __key__ = Provider(99)


Finally It turned out the error was absolutely justified...
In:

.addProjection(new PropertyProjection("last_modified", String.class));;

Last Modified was a Long and not a String...

Sunday, July 28, 2013

Suggested improvement for nodejs LOAD_AS_DIRECTORY module loading algorithm

In  the process for module loading is described.

on LOAD_AS_DIRECTORY(X) it says


1. If X/package.json is a file,
   a. Parse X/package.json, and look for "main" field.
   b. let M = X + (json main field)
   c. LOAD_AS_FILE(M)
2. If X/index.js is a file, load X/index.js as JavaScript text.  STOP
3. If X/index.node is a file, load X/index.node as binary addon.  STOP
It is nice to be able to require a folder name and have it automatically try to load index.js,
however when editing multiple modules having multiple index.js files open might become confusing or at least uncomfortable.

I wanted to suggest to add 


4. If X/LAST_PATH_FRAGMENT(X).js is a file, load X/LAST_PATH_FRAGMENT(X).jsas JavaScript text.  STOP5. If X/LAST_PATH_FRAGMENT(X).node is a file, load X/LAST_PATH_FRAGMENT(X).node as binary addon.  STOP

this way I can name the main js with the same name as the folder and stile just use require('mymodule') and it will load mymodule/mymodule.js.


Monday, April 29, 2013

Finding breaking changes in Adobe Cordova for PGB users

I have encountered a problem with Phonegap Build where some of my code stopped running.
Turns out there was a breaking change:


Before Cordova 2.2.0, the Connection object existed at: navigator.network.connection.
To match the spec, this was changed to navigator.connection in 2.2.0.
navigator.network.connection was left in place but was deprecated and was removed in 2.3.0.

I was not able to find any single place to read about such breaking changes.
see my Q to the people at phonegap build here:
http://community.phonegap.com/nitobi/topics/phonegap_version_upgrade_checkup_list_for_pgb_users

Eventually I ended up with the following Google search:


site:docs.phonegap.com ("API Changes" || "API change")

I hope this will help some one else.
If you know of a better way to keep track of breaking API changes - please leave a comment.

 
Clicky Web Analytics