Kotlin: A better Java

I’ve been quite vocal in the past that the Java language syntax is low on my list of favourite languages. I believe this stems from the fact that I write lots of C# (and even Typescript) these days, and those languages are a lot nicer to work with.

Lately I invested a little time in other languages, Go and Kotlin for the JVM.

While I’m yet to formulate my view on GoLang  I absolutely fell in love with Kotlin from jetbrains; if you work on the JVM I encourage you to check it out, It is simply awesome!

Effective Software Development 2017

I’ve been thinking a lot about the state of software development recently and moreover taking a step back trying to identify and articulate where I stand on past, present, future trends or concepts and define how to employ these ideas and approaches to become more effective in delivering successful production software.

Now I’m as guilty as the next person of Resume Driven Development, for years I’ve chased that shiny new technology; why? well it was current, it was bleeding edge or at least cutting edge, I gorged on technical articles and videos and spend many a long night trying things out, in fact, I still do and I can’t see that ever changing.

Experience vs. Skill vs. Attitude

However this can lead to having a lot of experience, but experience is cheap (as all that’s needed is time), it doesn’t necessarily lead to skill, this is harder this you have to practise at but a lot of people can do that. But attitude.  You either have it, or you don’t.  The right sort of person is so passionate about coding that can’t be stopped from doing it.  They typically started before high school and never looked back.  They write everything from assembly to JavaScript, on PCs to mobile phones, doing hard core computer graphics to high level social networking. They’ve tried everything!

For these type of people:

  • Learning becomes easier – The more one learns the easier new information takes root, it’s simple, you’re training your brain, you’ll find that you will develop an insatiable appetite for yet even more information, for tips, tricks, patterns, practises, anti patterns, frameworks… you name it…
  • Value identification – Because Netflix, Amazon, Microsoft [insert big name brand here] says it is shiny is no longer good enough! We’ve all been through this technology hype cycle where, something new comes out, everyone is really excited then one starts using it only to fall into the trough of disillusionment.  
    image
    Once you’ve been through this cycle a few times you know what to expect and you will better qualify where you lie on this trend line when evaluating for example a new technology stack.

 

My Views

Tooling

I grew up in Microsoft Land, if you take away my tools you’ll take away my productivity. Tooling in my view is critical to being productive. But tooling can be bad, tooling if not understood can abstract you so far away from the underlying technology you may not understand how it works, this could make a really trivial problem very hard to resolve.

The CTO of Expensify once wrote an article something along the lines of “Why I will never hire a .NET developer”. From what I recall he faced some serious backlash on that post, but I understood the point he was trying to get across. I have watched many Asp.NET webforms developers struggle when that community started embracing ASP MVC. For those that don’t know ASP Web Forms to some degree abstracted away web applications so much with the likes of ViewState and Postbacks, that a junior dev would be forgiven for thinking he/she was writing a VB6 application.

The point I’m getting to is that, if you understand how the underlying technology works, then you are in a much better position to leverage any tooling that increases productivity.

DevOps

In my view everyone should be aware of what DevOps means, for a long time developers worked away in their own individual boxes and likewise operations, only to knock heads and point fingers when trying to deploy software. I don’t think the industry can afford to continue with this approach, everyone developing software should be at least aware of what is involved in operations, only by seeing the struggles from another viewpoint can we all work together to streamline this process and improve our products and processes.

Language X versus Language Y

I’ve forgotten more languages than I know at this stage, but what I do know is languages are easy. I think I'm qualified to say that after writing a few applications in objective-C (can I remember much of the syntax? Nope! but I have the github repos to prove it! and give me a few hours and I'll remember it all).

What is hard with a language is; its eco system and frameworks, therein is where the effort lies. I grew up in C++ and loved it I gorged on it however when .NET arrived on the scene I knew its days were numbered (at least for the type of applications I was interested in). To this day C# is my favourite language. That said I also use Java day to day, and while I firmly believe it’s a less coherent language, the java ecosystem is much richer than its .NET counterpart. Which would I choose for a new project? This depends on the task I’m trying to achieve but moreover the team with whom I’d be working with and their technical capabilities. It’s good to expose yourself to different languages which will force you to learn different approaches to achieving an end goal, be a polyglot!

Try pick the Best language for the Job, keeping in mind there are many interpretations of what best means!

Static vs Dynamic Languages

A few years ago, after I’d heavily invested in my Javascript knowledge, I ended up writing a high level design document for a rather sophisticated Web application framework, I distinctly remember deliberating to no end on whether to insist on JavaScript or Typescript, the exact phrase I remember leaving my lips was “My heart says JavaScript, my Head says Typescript”, well with Hindsight, I’m glad to say my head was proven correct, I ended up prototyping and working on the initial version of that product and I firmly believe that Typescript has made large scale web applications more tolerable. I think dynamic languages are great for POC’s and early versions of products, why you can nearly type in the code and see the results in real time. However dynamic languages are harder to support, tooling is more difficult and a compile is your first unit test!

SOAP vs REST

Unfortunately, there is a lot of misinformation and misconceptions around REST, in fact they can’t really be compared directly given one is a protocol and the other an architectural style. But If i was to go out on a limb on the general expectation of what this comparison means, then I would say choose REST.

I listened to a talk from the Uber CTO talking about what he learned after writing 1000 microservices, and one of his points was the REST approach requires tooling that supports building and documenting the API. One such example I have experience with is Swagger. Tooling always wins out in my book, I don’t want if at all possible to have to learn the Swagger syntax, much the same way for these last 20 years I’ve never explicitly learned WSDL, once I understand how it works and what it’s for I’ll use a tool to generate it faster and with less errors that I could ever hope to achieve.

Clouds…

For years I’ve been saying, “bet on the web”, now I say “bet on the cloud”.

I think public clouds are the single biggest game changers in todays software industry, by lowering the barrier to entry, companies can get off the ground quickly, with limited resources and short times to market. My preference for Azure (where possible) is no secret.

A client of mine once told me how back in the .COM boom he was paying 100k $ a month to have a geo-redundant solution that involved a container up some mountain some where in the US. With the cloud the infrastructure already exists, and with a flip of a switch it’s possible, sometimes with no changes to the underlying application itself.

But sadly cloud can be used in all the wrong ways and the costs can quickly ramp up. I would suggest the following guidelines when looking at the cloud

  • If you are lifting and shifting, then you are probably doing it wrong
  • If you stay in IaaS you are missing out on the advantages of PaaS
  • If you use serverless, then use it correctly, don’t turn your application into one giant RPC call just to be sexy.

Understand your Business

You may have very smart people in your business that can’t innovate as they don’t understand the big picture, identify these people and ensure they are in a position to contribute. Developers and business need to speak the same language if innovation is to be expected.

Be aware of the innovators dilemma and don’t stagnate,.

Microservices

How many times have you heard people moving to microservice architecture to escape the problems they have with their current monolithic approach?

Yes microservices can help one avoid some of the pitfalls of monolits however I would argue that if someone can’t write a good monolit then they won’t write a good set of microservices. Some backgrounds like the Microsoft stack naturally fit well into Microservices while other like J2EE are in stark contrast.

The Promise

Microservices are certainly one of the areas where people encounter the hype cycle in early attempts. They break up big application in the hope of:

  • having greater flexibility and scalability
  • having to ability to deploy services independently with different teams
  • have more agility

The Result

  • It is more brittle than ever before
  • Performance is terrible
  • Services needed to be deployed together and in specific order
  • It was impossible to follow the flow of messages through the system.
  • Distributed systems are HARD
  • Eventual consistency is a paradigm shift
  • Legacy habits created a distributed “big ball of mud”

I’m not looking to frighten anyone, I’m pro microservice but one needs to understand what they are buying into and what approaches exist to overcome these limitations.

Architecture

Architecture is never just about the technology, it’s about the team, are they agile, do they “do agile” or neither, is there a DevOps culture, what are the underlying business decisions driving the change?

Keep the follow in mind when architecting a solution:

  • Keep it simple
  • Don't build what you don’t need
  • Don’t build what you might need
  • TCO and ROI are important
  • Research new approaches,
    e.g. I’d rather drink battery acid than do 2 phase commit, there are other approaches…. research them!

 

Always be learning!  

Powershell Oracle Db Backup

Hi all,

I thought I would share with you this quick Powershell script I created to Export an oracle database, 7Zip it and then upload it to AmazonS3

Export.ps1

param([String]$dumpname=(get-date -format dd-MM-yyyy))
          
Set-Alias sz "$env:ProgramFiles\7-Zip\7z.exe"
          
expdp <myDbUser>/<mypasswd>
   DIRECTORY=dmpdir DUMPFILE=$dumpname.dmp
   LOGFILE=$dumpname.log
         
sz a -mx "$dumpname.7z" "$dumpname.*"
Write-S3Object -bucket "company-ps"
   -profilename brian.keating
   -file "$dumpname.7z" -key "customer/dbdumps/$dumpname.7z"
remove-item "$dumpname.dmp"
remove-item "$dumpname.log"

       

 

How it works:

It uses the current date as the dumpname variable (unless specified as a parameter to the script)

It uses Oracle DbPump to export the database (assumes 7-Zip is installed in %\program files\)

It then writes the 7z file to AWS S3

Finally it removes the dump and log files.

AWS Powershell

ASW Powershell is installed on this server and I have already set a profile “brian.keating” with the following command.

set-awscredentials –AccessKey <ACCESSKEY> –SecretKey <KEY> -storeAs brian.keating

 

Trivia

The server in this case is running in AWS, the 7zipped db is about 4Gb, and it uploads to S3 in about 30seconds (nice!)

Azure Logic Apps– Lesson 1 - Quick Sample

 

Hi all, I’ve been pretty busy lately and have been struggling to get the time to write a few blog posts. So what I’ve decided to try do is create a few videos instead, I’m going for a warts and all approach so I’m not editing the videos before publishing, this will allow you to get a better feel for what is involved in working with Azure rather than looking at a well polished demo that leave some critical parts uncovered (that’s my excuse for not making time to try edit it!)

Lesson 1

A quick sample to get a feel for Logic Apps.

Out with the old in with the new(er)

With 2016 drawing to a close and 2017 already in full swing for me, I thought this was a good opportunity to reflect on how 2016 went and what 2017 has in store for me from a technological point of view.

2016

If asked how 2016 was from a professional perspective I’d probably try to sum it up as follows “Technology continued to roll out at an ever increasing pace, not only was new technology appearing faster than ever before, existing technology stacks started iterate and churn under our feet!”

Nearing the end of 2016 was where I finally admitted defeat and realized that I can’t keep up with everything and I while I sure am greedy and to know everything about everything, it was getting to the point that I was becoming a ‘Jack of all trades and master of none’ dare I say a full stack developer! I’d actually like to think I’m master of some, but certainly it was a big effort to stay on top of everything.

What did I get up to?

Azure : I got certified in Azure this was without doubt my most prized professional achievement of 2016, I’ve been using Azure for years and I feel quite confident in acclaiming it to be the best public could in the world today.
I’ve also started work on a state of the art data distribution network using Server-less architecture. I finally got down and dirty with Swagger/API Apps/LogicApps/AzureFunctions.
I got a lot better at networking, Load balancing resiliency, Azure/AWS causes a, devops inner persona to ooze its way to the top.
I listened with baited breath to the Azure Weekly Azure Podcast to see what was new (and always scratched my head when Cale got excited about BlockChains, perhaps next year I’ll look back and kick my self for not being an early adopter, it does seem to be an area that’s heating up).

AWS: I got certified as an AWS Solutions Architect, it was great to get a better understanding of AWS and indeed for a few offerings they I’d choose them over Azure. Got heavily involved in AWS CloudFormation and helped regain some control on AWS madness.

Google Cloud: I spent a few weeks playing with it just to see how it’s coming along, at least now I’m somewhat informed but I’d only consider myself as a beginner (I’d consider GoogleCloud a beginner also , unless they put in a massive investment into the portal and services, they simply can’t compete with Azure and AWS.

Docker: I can create images, start stop then understand volumes, I didn’t get as far as any of the clustering techniques such as swarm but I see huge value in Docker!

AngularJS: Architected and delivered a cutting edge data visualization system based on Angular 1.x, typescript,sccs,gulp. 
Introduced AngularJS in to multiple smaller projects.

Typescript: This is a fantastic language, and now especially with all the bells and whistles in v2.1 (not least async/await for es5 targets). If you are writing any Javascript you need to learn this no one will ever convince me that a dynamically typed language is better than a statically typed language for starters, but with all the new Standards based features now baked in, it’s certainly taking the industry by storm, I can’t see how Babel will continue to fight for its place in the world alongside it.

Ionic2: I wrote another mobile app, I’ve done this in many languages to date, I started out with iPhones and xamarin c#, moved to objectiveC and java, and finally settled back on the Typescript/Angular2 based Ionic2 framework. It’s a pleasure to deal with, and with my other investments in the underlying stack it has become a natural fit.

Java8: Finally spend some time getting up to speed on the new jdk and it’s offerings. While not strictly Java8, I’m including Sprint Bootstrap, Wildfly10 Application server, CDI, JaxRS etc in this section.

Camel: Gained a basic understanding and working knowledge of the Camel EAI framework.

ActiveMQ: I debated about putting this one on here, all queues fulfil the same core requirements to pass messages right? But I did approach ActiveMQ from three different sides camel/c#/java, so that was interesting.

.NET 2017 I’m now informed about what’s coming down the line. Some interesting thing like C#7 (which I will admit I had to read twice before I saw the value in the language changes), better support for the web stacks (although I’d admit with a tear in my eye that I’ve moved to Jetbrains software and am unlikely to come back to VStudio unless it’s an ASP based backend).

Client Products: Not only the development stacks have been changing, products in use by my clients have been moving at a rapid pace also and given they pay the bills I dedicate a fair amount of time to understanding them in depth.

Resource Consumption:
DNR -Listened to nearly every episode of DotNetRocks.
Hanselminutes - Funnily enough I found DotNetRocks as I used to subscribe the Hanselminutes; I say used to, as I’ve finally given up on Hanselminutes it appears to have moved in a different direction in the last year or so, don’t get me wrong, Scott is a great guy one of the best technical speakers in MS if you ask me, I even follow the weekly ASP.net stand-ups which he’s in, it only the podcast that I gave up on in December.

Other recommended podcasts:
Angular In Action
Javascript Jabber
RunAs Radio
Azure Podcast

2017

As you can image it takes a lot of time to get proficient in any of the above stacks I’ve mentioned . I’ve been trying to stay on top of them all and I’ve now reached the point or realization that need to let some go (think of Kate Winslet prying Leonardo DiCaprio’s icy fingers off that board she was on, it’ll be oh so sad). I’m going to narrow down the field, I’ll still keep in touch with them and if I encounter anything I don’t understand then I’ll make it my business to understand it, I simply won’t actively go pursuing them all. I’ve been burnt before with that approach, I learn Sliverlight after all, it wasn’t all bad as I wrote a windows phone app and many WPF apps around the same time, so the experience transferred nicely, it’s just that I’m not writing much WPF these days so I’ll put effort back in that direction only if and when needed.

Q: So the question remains: Where am I going to put my extended effort this year? 
A: Azure first approach. Azure will be the primary topic of my blogs whether the implementation is in C#/JavaScript/Typescript/Java I don’t really care. If the backend is .NET or Java, again I don’t really care but I do intend on blogging on practical usage cases for Azure services, I may even create a video or two!

Happy new year!

Tech of the week

Appears to be the story of my life lately, just as I get excited and proficient on one stack I get side tracked with something else. The latest stack I’m playing with is the Ionic2 framework, but first some history.

 

UX Stacks

I’ve written UX applications in many different way in the past

  • java swing (albeit i’d never put this on my resume as I can’t recall a single bit of that application i helped a student friend with so many years ago)
  • MFC – Oh the pain
  • ATL/WTL – Actually i quite like this back in the old C++ days when men were men.
  • C# Windows forms – Spent years on this stack and if you’re happy with battleship grey it’s still quite RAD
  • WPF/Silverlight – Wrote quite a few applications in this, probably wouldn’t call myself an expert but I can xaml like the next guy
  • MVC3+ – Have written and still support a quite a few MVC applications it was the bridge that finally moved me heavily to web client side tech.
  • Objective-C/XCode – A handful of iPhone applications, it’s ok, language is a little weird yes but use it for a week read a few books/material and you’ll make a good stab at it.
  • Xamarin – A nice approach for writing mobile apps in  c#, problem is you still need to use the native designers (yes i know about xamarin forms.. are we really going to talk about that?)
  • Web tech, My transition to the web was a natural progression that leveraged things I’d learned with other frameworks, e.g Knockout was familiar from wpf data binding, angular has somewhat a familiarity to MVC

Ionic2

So what is Ionic2? In short it’s a framework for writing mobile (and progressive) apps. As you’ve seen above I’ve already ways of writing mobile apps, So why something new? Why not! in this world – always be learning!

In my case there were some more compelling reasons,  see for the last year I was architect and hands on developer on a new web application using angularjs v1 sass, typescript and a tonne of wonderful libraries. Ionic2 is built around the same stacks, applications use angular2, html5, typescript, saas, gulp etc, the list goes on and on. Apparently I’m finding my home in this world, in fact if i was to write a greenfield desktop application I’d seriously consider writing it with html tech and framing with Atom/Electron.

Sample app

image_thumb2 Taking the application on the left as an example,
I was to create this screen in a few hours, sure i’s not the prettiest and contrasts are high, but the point is that from a blank canvas it only took a few hours to design and implement this screen using Ionic2 and web technologies. I’m a little spoiled as I can somewhat ignore the stubborn elephant in the room *CSS*. i’m targeting the latest devices so I’m getting to use flexbox for layout , css3 transitions etc.

I’m seriously convinced that when it comes to writing general purpose mobile applications Ionic2 will be hard to beat.

I know if i was to write the same screen in xcode/android studio it would take me a lot longer to implement this design.

Check out Ionic2 for yourself you won’t regret it.

Azure vs. AWS Text to Blob with SDKs

This demonstrates what is involved in writing and reading some text to an Azure and an AWS blob.

Use case

What i set out to achieve was to demonstrate how to read and write some text to a blob with the SDKs. Just to make it a little more interesting I decided to use .NET for the reading and Java for the writing.

Obtaining the SDKs

Adding the SDKs was a seamless process, for .NET Nuget was used and for Java Maven was used

 

.Net Java
image
image
image

 

Write

Azure

image

 

AWS

image

 

Read

Azure

image

 

AWS

image

Conclusions

Both SDK’s were trivial to install and use, the Azure SDK’s suited my use case a little better in that they didn’t need me to deal with files in my Application code (I expect text is not a mainstream use case).

AWS as always relies on the region being specified which I can’t say I like that much.

Certification

It’s been a busy year and my blog posts have suffered while I’ve been preparing and sitting these exams. But I’ve got them both now, so expect my Azure vs AWS series of posts to continue.

signature

Media Indexing In the Cloud

So out of the blue I found myself giving Azure Media Indexing a trial run, for no other reason other than I could, this is why I love cloud tech so much, it brings something that would have been very difficult 5-10 years ago, within reach of anyone with an public cloud account.

AWS vs Azure

Both AWS and Azure have media services, typical used to manage digital media and serve it up to consumer playback devices at scale.

AWS has the the Elastic Transcode and Azure has Azure Media Services, however only Azure has the ability to dig into audio or media files and extract the text within.

Azure Media Indexer

Azure Media Indexer enables you to make content of your media files searchable and to generate a full-text transcript for closed captioning and keywords. You can process one media file or multiple media files in a batch. Have a look at this post for some details on how to do it from code https://azure.microsoft.com/en-us/documentation/articles/media-services-index-content/

The code uploads a file then starts an indexing job, then downloads the results:
Note: The source code seen above has a typo, I’ve submitted a pull request to hopefully this will be fixed, but easy to spot.
Also the download part failed with an exception for me so i just pulled it down with a little bit of code on a second pass.

image

 

The above code is possibly all you need if you wish to upload content and start the indexing job manually with the old portal.

Here’s how:

On your media account upload some content

image
image

 

Once the content is uploaded, start the indexer process, set a good title as Azure will reach out to the interweb and use it to seed the language extraction.

image
image

 

There is no way to download the output from the portal so use use the code i shared above to download the content.

I processed the latest podcast at time of publishing from https://www.dotnetrocks.com/
https://s3.amazonaws.com/dnr/dotnetrocks_1276_news_from_build.mp3 

In hindsight it was possibly not the best podcast to index as it was recorded live @build (i expect, i’m two episodes behind on DNR this week so have not listened to it yet), the DNR guys typically have exceedingly good audio, at some stage it might be worth indexing another episode.

Results

You can find the results here here , initially my knee jerk reaction was , “ah this is poor” but after reflecting on it I’m blown away by what was done and so so, sooo easily!

With a bit of editing this can be thrown into Azure Search / Sql Server etc for full text search and direct seek media playback.

See for yourself:

image

 

For sure it needs some editing, e.g.
I release the eleventh music decode by
should in-fact be
I released the eleventh music to code by

but what a great start!!!

Cloud costs: Shut those VM’s down

The public cloud is fantastic for numerous reasons, if you’re not faced with some restriction such as where you data lives or other factors, then my advise is get away from private clouds and get to the public clouds as fast as your legs can carry you!

However once you’re there it’s not all plain sailing, if you let a team of people loose to play with with all these new toys, on the back of your company’s credit card, then costs can start to accumulate very quickly!

Sometimes, your VM’s are not being used for production and what invariably happens, is that these machines get forgotten about or are left running for no good reason, now while there are a few ways to capture such scenarios,  what I’ll show you now is a very quick way of scheduling those known VM’s to shutdown (or start up) as on a predefined schedule,

AWS

For AWS the easiest way of scheduling a single standalone VM to shutdown is to use the AWS Data Pipeline service.

image

Lets quickly show the workflow:

1) Create new Pipeline with CLI Command

image

 

2) Enter the Stop EC2 CLI commands

image

Note: This field only shows as one line of text vertically in chrome so I modified to styles to show the full command.


You can see that i have two different stop commands, I could combine these into the one command with the two IDs however if one fails then they both fail, this can be problematic if for example an Instance gets terminated.

3) Schedule

image

 

4) Set log file bucket

image

 

5) Select role

image

Choose custom and then select the two defaults.
Security Note:  Roles needs to be configured to allow Data Pipeline access to your VM’s, please see here: https://aws.amazon.com/premiumsupport/knowledge-center/stop-start-ec2-instances/

6) Done

image

That’s it, you now have a scheduled task that will switch off your vm’s nightly. It should be noted that this will start a micro data pipeline ec2 instance VM with a default run time of 50 minutes, so you need to ensure the end justifies the means, better yet reduce the run time by editing the workflow to e.g. 15 minutes.

image

 

Azure

In order to achieve the same results with Azure we are going to select Azure automation,

image
If you’re familiar with Azure you will know that there are currently two ways of creating VM’s, the classic approach and the RM (resource manager approach). In this post I’ll show you the RM approach, but feel free to substitute classic in it’s place with a nearly identical approach.

1) Open or create an Azure Automation account.

image

 

2) Edit Assets

image[95]
image

Add a variable for the AzureSubscriptionId you’ll be using
Select your service principle account, you’ll have to search for it to appear.

3) Runbook

We have two options now, we can either use some powershell or some graphically defined workflows, let’s do this with a graphical version, we don’t need to create this, we simply import from the gallery.

image
image

After importing choose Edit on the runbook

image
image

4) Set inputs

image

Then we set the two Assets we provided earlier and optionally a ResourceGroupName (to stop all vm’s in a resource group) or a VMName The “Auto” you see above isn’t a keyword, it’s my badly named ResourceGroup.

5) Publish

image

 

6) Set schedule

Go back to the Runbook and choose schedule

image

With the schedule you can specify any of the input parameters and override the defaults if you so wish.

Security Note: Much the same as Azure you’ll need to ensure you’ve permission to access the VM’s from Azure Automation, the best option is to create a SecurityPrinciple application. See: https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/

 

Conclusion:

While it does look like the Azure approach is much more convoluted it is much more powerful, e.g. it is very easy to extend the azure run book to check all VM’s for a “Production” tag and only shutdown vm’s if they are not production (because that would be bad right!); with AWS, we are simply relying on a feature of Data Pipeline that allows us to run simple cli commands.

Pricing is much of a much-ness between each, with Azure you can run for free (to a limit)

image

AWS the 15 mins with a micro instance is not even worth worrying about.

Recent Tweets

Note: For Customization and Configuration, CheckOut Recent Tweets Documentation

Calendar

<<  December 2017  >>
MoTuWeThFrSaSu
27282930123
45678910
11121314151617
18192021222324
25262728293031
1234567

View posts in large calendar

Month List