Closing notes
Unfortunately , we reached the conclusion that HBase is still not stable enough for us to use in production environment. I think it’s a great project , and I...
Posts by Tag
- Hadoop 14
- Hbase 14
- Scala 101 5
- Java 4
- C# 2
- Maven 2
- Tomcat 1
- Hibernate 1
- Mockito 1
- Unit tests 1
- Off Topic 1
- Rants 1
- Log4Net 1
- jar 1
- code quality 1
- XML 1
- Groovy 1
Hadoop
Bug Update
After a lengthy discussion with St.Ack (on the hbase channel in the mirc) and Jean Daniel about this bug , we currently believe that what we’re seeing is “ …...
Can’t live with it
We assumed that the bug was caused mainly because of the high load rate , and that once the bulk of the data will be in HBase and the load will drop consider...
We’ll try to live with it
As you remember , HBase tends to collapse (Return”NotServingRegionException”) after a few millions of files (Latest crash: 6 million). Since we (want to) bel...
Still unresolved
We’ve been working on the previous bug (last post) for a few days now , but unfortunately nothing is working. We’ve suspected the error might be caused by wr...
First Setback
We’ve tried loading the system with 10,000 , 100,000 and 200,000 files - everything worked perfectly.
Begin Experimenting
Today we started the production-mode experiemnts.
Testing update
We are planning to move on to production soon , and we intend to build a test environment that will resemble the production environment as close as possible.
Hadoop Failover and Failback Solution
As you can see in the previous blog , Hadoop successfully failover , but it fails to failback. Our solution to this problem is this:
When will hadoop NN recognize a failure?
I’ve been trying to cause the NN to recognize that it is inconsistent and turn to the SNN for data.
Hadoop failover (and hopefully failback)
We’ve decided to test using linux HeartBeat together with hadoop , to enavle failover (and failback) capacbilities.
Testing Hadoop - Starting HBase
Hbase configuration and running is very similar to hadoops. Not surprisingly , they also have a nice Getting Started page. The tricky part , though , is to u...
Testing Hadoop - Starting Hadoop
First thing : It works. If you’re getting a lot error messages and you start thinking “well , maybe it’s crap. still 0.18 can’t be that good a version” Stop...
Testing Hadoop - Problem Definition
We have a very large amount of relatively small files (~5k avg , 41k max , 0k min) , that we access a lot (20M times a day) for various computations. Current...
Hbase
Closing notes
Unfortunately , we reached the conclusion that HBase is still not stable enough for us to use in production environment. I think it’s a great project , and I...
Bug Update
After a lengthy discussion with St.Ack (on the hbase channel in the mirc) and Jean Daniel about this bug , we currently believe that what we’re seeing is “ …...
Can’t live with it
We assumed that the bug was caused mainly because of the high load rate , and that once the bulk of the data will be in HBase and the load will drop consider...
We’ll try to live with it
As you remember , HBase tends to collapse (Return”NotServingRegionException”) after a few millions of files (Latest crash: 6 million). Since we (want to) bel...
Still unresolved
We’ve been working on the previous bug (last post) for a few days now , but unfortunately nothing is working. We’ve suspected the error might be caused by wr...
First Setback
We’ve tried loading the system with 10,000 , 100,000 and 200,000 files - everything worked perfectly.
Begin Experimenting
Today we started the production-mode experiemnts.
Testing update
We are planning to move on to production soon , and we intend to build a test environment that will resemble the production environment as close as possible.
Hadoop Failover and Failback Solution
As you can see in the previous blog , Hadoop successfully failover , but it fails to failback. Our solution to this problem is this:
When will hadoop NN recognize a failure?
I’ve been trying to cause the NN to recognize that it is inconsistent and turn to the SNN for data.
Hadoop failover (and hopefully failback)
We’ve decided to test using linux HeartBeat together with hadoop , to enavle failover (and failback) capacbilities.
Testing Hadoop - Starting HBase
Hbase configuration and running is very similar to hadoops. Not surprisingly , they also have a nice Getting Started page. The tricky part , though , is to u...
Testing Hadoop - Starting Hadoop
First thing : It works. If you’re getting a lot error messages and you start thinking “well , maybe it’s crap. still 0.18 can’t be that good a version” Stop...
Testing Hadoop - Problem Definition
We have a very large amount of relatively small files (~5k avg , 41k max , 0k min) , that we access a lot (20M times a day) for various computations. Current...
Scala 101
Scala 101 - Scoping
This is a part of my Scala tutorial . Read the first part & second part for a more general Scala intro. You can read here for an overview of how to write...
Scala 101 - Named & Default parameters
Another syntax sugar provided in Scala are the default values for parameters , and the named parameters. Its fairly stright forwards , so We’ll just start wi...
Scala 101 - OOP : Getters & Setters
This is a part of my Scala tutorial . Read the first part & second part for a more general Scala intro. You can read here for an overview of how to write...
Scala 101– Basic OOP : Writing a class
This is part 3 of my Scala tutorial – read the first part and the second part for a more general Scala intro. All the examples you see here were ran via the ...
Scala 101 - Val, var, Arrays, Loops and Conditions
Today I’ve started a 3 days course (sponsored by my work place ) in Scala. The prerequisites are just”Experienced in Java” .
Java
XMLing in Java? Use java-xmlbuilder
XML is not native to Java. Many will disagree, but this is just the simple truth. It’s tedious, messy and generally ugly - specially compares to Groovy or Sc...
Mockito! Your unit test side-kick
Problem:
LinkedHashMap
Problem:
C#
Using Log4Net in a C# web application
I came to Log4Net because I really loved Log4j , but I must say the documentation on this project is simply crappy, and half of the links in the project page...
Learning C# with a Java background
I’m a Java programmer , and I’m starting to work on a production project written in C# , so this is a great time to start learning this language . I’ve neve...
Maven
Maven: Building a Java-Groovy mixed project
I have a project that mixes Java and Groovy, and the main problem I ran into was that the jar was nicely built, but it only contained the Java classes, and n...
Maven - Creating an executable jar
I wanted to create an executable jar in maven , s.t whenever I run”mvn install” , it will generate both the regular one and the executable one.
Tomcat
Back to Top ↑Hibernate
Back to Top ↑Mockito
Mockito! Your unit test side-kick
Problem:
Unit tests
Mockito! Your unit test side-kick
Problem:
Off Topic
Missing the whole point of a password
(Off topic)
Rants
Missing the whole point of a password
(Off topic)
Log4Net
Using Log4Net in a C# web application
I came to Log4Net because I really loved Log4j , but I must say the documentation on this project is simply crappy, and half of the links in the project page...
jar
Maven - Creating an executable jar
I wanted to create an executable jar in maven , s.t whenever I run”mvn install” , it will generate both the regular one and the executable one.
code quality
Why you should never ask permission to clean up code
http://cdevroe.com/notes/code-maintenance-dont-ask/
XML
XMLing in Java? Use java-xmlbuilder
XML is not native to Java. Many will disagree, but this is just the simple truth. It’s tedious, messy and generally ugly - specially compares to Groovy or Sc...
Groovy
Maven: Building a Java-Groovy mixed project
I have a project that mixes Java and Groovy, and the main problem I ran into was that the jar was nicely built, but it only contained the Java classes, and n...