MongoDB Cluster Switching: Leveraging TXT and SRV Records
Posts by Year
- 2023 3
- 2022 1
- 2020 6
- 2019 10
- 2017 1
- 2016 3
- 2015 2
- 2014 4
- 2013 4
- 2012 3
- 2011 13
- 2010 7
- 2009 6
- 2008 12
2023
Querying Mongo in C#
In this blog post, I’ll show so example on how to use C# MongoDB drive to querying a collection called “movies”, available in the sample data provided by Mon...
Finding AWS Service Names for the Cost Explorer Command
AWS Cost Explorer can be a powerful tool for monitoring your AWS spending, but it’s not always easy to use. One common issue is finding the correct AWS servi...
2022
Serverless Security 101: How to think about serverless cloud security?
Switching to - or even better, starting with - serverless infrastructure is great, especially when you’re a small/medium company. Your expenses only grow whe...
2020
More than just event bus - EventBridge drill down
A few months ago we discussed which AWS service you could use to deploy a Pub-Sub solution and mentioned EventBridge as a leading candidate. But EventBridge ...
Infrastructure outages happen. Be one step ahead.
Sometime last Wednesday we started getting weird errors from our systems. Everything was up, the logs weren’t reporting any abnormal behavior, but some thing...
Asymmetric JWT Signing using AWS KMS
TL;DR Node.js implementation using asymmetric encryption key stored on AWS KMS to sign JWT tokens and verifying them using the public key. You can skip to th...
Pub-Sub Services on AWS: What are your options?
Pub-Sub architecture
Automating Cross-Account Role creation to access users’ account
Operating within users’ accounts is a serious matter. We want to make sure that we get all the credentials we need to deliver great user experience, but limi...
Creating users on Serverless Aurora/Mysql
As part of our new deployment, we started testing AWS’s Aurora Serverless - which is basically RDS on demand to use from our lambda functions.
2019
Debugging Spring Rest Requests
We’ve been working with Spring Boot for a while now, and it gets the job done nicely. However, sometimes, somewhere between the model annotations, the defau...
Default Jackson serde in Spring
Problem We’re using Spring in most of our Java applications, and rely on Swagger for a lot of API calls. Now, having all the jackson annotations makes everyt...
Lambda: Move S3 file based on content
We have a process that saves a file to an S3 bucket. We needed a lambda to read the file, parse part of the content, and move the file to the appropriate fol...
Basic Lambda to call internal VPC api
Product team decided they wanted a specific event to happen every time a specific email address receives an email. The first option was to poll mail server a...
Some kafka retention lessons (we learned the hard way)
We’re using Kafka (2.0 on cluster, Java client 1.1) as our messaging backbone. A different cluster is deployed in each environment, and we recently started s...
Postgres - Logging queries
We had an issue with some JQL queries returning weird results from the db, so we wanted to see exactly what’s arriving to the psql service. To see that:
AWS Re:Invent 2018 – My Top Sessions
SVR322 - From Monolith to Modern Apps: Best Practices We are a lean team consisting of developers, lead architects, business analysts, and a project manag...
AWS Re:Invent 2018 - Recap
On November 2018 I was on my first AWS Re:Invent convention in Vegas. This was an amazing experience, which I highly recommend to anyone working with AWS (do...
Kafka Streams runtime exception handling
How to handle runtime exceptions in Kafka Streams
AWS Certified Solutions Architect - Associate
I’ve been doing a Udemy course as a preparation for the AWS Certified Solutions Architect - Associate. These are my summary notes
2017
Monitoring CloudWatch statistics using Graphana, InfluxDB and Telegraf
We’ve started checking out monitoring solutions for our AWS-based infrastructure, and we want it to be not-that-expensive, monitor infrastructure (cpu, I/O, ...
2016
Two JS lectures that will change the way you thing of JS
I’m a backend guy, with backend habits and natural aversion from JavaScript. I used to think of it as the frankenstein of language - something hastly create...
Adding badge icon to ionic Android app
TL;DR You have an Ionic application, and you want to use icon badges on Android. You’ve found the cordova badge plugin, but it only works if the the applicat...
How lucene’s mlt works
###TL;DR It takes the document, computes tf/idf, takes the top 25 (configurable) terms, and queries the DB with these terms with a boost.
2015
Port forwarding on ec2
we want to get to prod1 but it’s only accessible through mid and we want to reach it from localhost
Apple Provisioning Hell
We’re developing an ionic app for iOS and android, and handling the Apple provisioning/certificates/apn profile is hell. Just pure hell.
2014
Good NodeJS habits and practices (Heroku)
Heroku just published in their blog a very nice list of best practices for working with nodeJS and npm. Great read!
Easy Executors and Callables in groovy
Multithreading and general multi-tasking in groovy is super easy, thanks to frameworks like GPars. However, even the basic Java frameworks can be easily util...
Url catching regexp
Not exactly a brilliant piece of engineering, but this is a useful dirty hack. If you want to clean urls from a string, you can match it using this regexp:
Node JS for backend programmers
I started working in a new and exciting start up, and the thought of using Java for development crushed my soul, so I began shopping around for a fast-develo...
2013
Git: Checking out only a specific folder
I’ve been looking for a way to checkout only a specific folder from a git repository. Most answers I found to this problem stated that it’s not feasible - bu...
Web programming for server people
I’m mainly a back-end engineer - through my not-so-long career, I’ve dealt with sql, no-sql, offline processes, etl mechanisms, billing services, web-crawler...
Finally! What’s hogging your RAM?
Some of you may have noticed that the Windows Task Manager doesn’t show you the real memory consumption of the applications you’re running. It’s quite obviou...
Zorpia is a scam
I’ve been getting a lot of emails lately from a website called”Zorpia” claiming I have a message from someone named Katja. I’ve never signed into this websit...
2012
Maven: Building a Java-Groovy mixed project
I have a project that mixes Java and Groovy, and the main problem I ran into was that the jar was nicely built, but it only contained the Java classes, and n...
XMLing in Java? Use java-xmlbuilder
XML is not native to Java. Many will disagree, but this is just the simple truth. It’s tedious, messy and generally ugly - specially compares to Groovy or Sc...
Beware of Doubles, Floats, and generally floating points…
I’ve recently encountered few people who were naive about the representations of real (i.e not integers) numbers in computer languages , like double and floa...
2011
Comparison method violates its general contract!
java.lang.IllegalArgumentException: Comparison method violates its general contract!
SearchIndexer & TSVNCache : Two things that are hogging your hd - and your computer
I have a brand new Lenovo T510 , i7, 4G ,64bit , Windows 7 with an Nvidia GPU - and it sucks. In this price , with these specifications , it really shouldn’t...
Scala 101 - Scoping
This is a part of my Scala tutorial . Read the first part & second part for a more general Scala intro. You can read here for an overview of how to write...
Scala 101 - Named & Default parameters
Another syntax sugar provided in Scala are the default values for parameters , and the named parameters. Its fairly stright forwards , so We’ll just start wi...
Scala 101 - OOP : Getters & Setters
This is a part of my Scala tutorial . Read the first part & second part for a more general Scala intro. You can read here for an overview of how to write...
Scala 101– Basic OOP : Writing a class
This is part 3 of my Scala tutorial – read the first part and the second part for a more general Scala intro. All the examples you see here were ran via the ...
How Sun planned to discontinue Java
Cool story about how sun almost ditched Java , and only by chance it became what it is today
Scala 101 - Val, var, Arrays, Loops and Conditions
Today I’ve started a 3 days course (sponsored by my work place ) in Scala. The prerequisites are just”Experienced in Java” .
Managing both Hebrew & English wordpress on the same host
I opened up a new blog about investments , and I wanted it to be in Hebrew. Problem is , WordPress is basically for ltr languages - and if you install the rt...
Why you should never ask permission to clean up code
http://cdevroe.com/notes/code-maintenance-dont-ask/
Swing - Closing the window and finalizing stuff in the background
I’m writing a program that is a little data intensive. At first , when the user clicked the X and choose”yes” at the”are you sure” box , I would do the clean...
Maven - Creating an executable jar
I wanted to create an executable jar in maven , s.t whenever I run”mvn install” , it will generate both the regular one and the executable one.
2010
SVN Working Copy xxx locked and cleanup failed
This thing drives me insane every time . It almost always happens in the middle of a mega-update, and usually the only solution is to delete everything and r...
Consuming SOAP in Groovy
I’ve started using SOAP in Groovy. As I’m new both to SOAP and to Groovy , this is a little basic.
10 Developers For The Price Of One
“Programming managers have long recognized wide productivity variations between good programmers and poor ones”
Grep on PowerShell
I must admit that I find the windows PowerShell a really-way-too-late pale substitution for the full capacity of the unix shell , but since I sometimes need ...
Using Log4Net in a C# web application
I came to Log4Net because I really loved Log4j , but I must say the documentation on this project is simply crappy, and half of the links in the project page...
Learning C# with a Java background
I’m a Java programmer , and I’m starting to work on a production project written in C# , so this is a great time to start learning this language . I’ve neve...
Missing the whole point of a password
(Off topic)
2009
Mockito! Your unit test side-kick
Problem:
LinkedHashMap
Problem:
Closing notes
Unfortunately , we reached the conclusion that HBase is still not stable enough for us to use in production environment. I think it’s a great project , and I...
Bug Update
After a lengthy discussion with St.Ack (on the hbase channel in the mirc) and Jean Daniel about this bug , we currently believe that what we’re seeing is “ …...
2008
Can’t live with it
We assumed that the bug was caused mainly because of the high load rate , and that once the bulk of the data will be in HBase and the load will drop consider...
We’ll try to live with it
As you remember , HBase tends to collapse (Return”NotServingRegionException”) after a few millions of files (Latest crash: 6 million). Since we (want to) bel...
Still unresolved
We’ve been working on the previous bug (last post) for a few days now , but unfortunately nothing is working. We’ve suspected the error might be caused by wr...
First Setback
We’ve tried loading the system with 10,000 , 100,000 and 200,000 files - everything worked perfectly.
Begin Experimenting
Today we started the production-mode experiemnts.
Testing update
We are planning to move on to production soon , and we intend to build a test environment that will resemble the production environment as close as possible.
Hadoop Failover and Failback Solution
As you can see in the previous blog , Hadoop successfully failover , but it fails to failback. Our solution to this problem is this:
When will hadoop NN recognize a failure?
I’ve been trying to cause the NN to recognize that it is inconsistent and turn to the SNN for data.
Hadoop failover (and hopefully failback)
We’ve decided to test using linux HeartBeat together with hadoop , to enavle failover (and failback) capacbilities.
Testing Hadoop - Starting HBase
Hbase configuration and running is very similar to hadoops. Not surprisingly , they also have a nice Getting Started page. The tricky part , though , is to u...
Testing Hadoop - Starting Hadoop
First thing : It works. If you’re getting a lot error messages and you start thinking “well , maybe it’s crap. still 0.18 can’t be that good a version” Stop...
Testing Hadoop - Problem Definition
We have a very large amount of relatively small files (~5k avg , 41k max , 0k min) , that we access a lot (20M times a day) for various computations. Current...