Welcome to the Workday Developer Blog. This blog does not discuss Workday API’s or how you can achieve task ‘X in the Workday system, that is already well served by the Workday Community. This blog provides a place where we can discuss technologies we are actively researching, using or just excited about.

Connect with us on Twitter @WorkdayDev

Performance Debugging as Exploratory Visual Analysis

By CC Kanne, Workday, San Mateo

Debugging production performance issues in distributed systems is rarely easy. A large number of factors influence system behavior. Controlling all of them to get deterministic, comparable results is usually impossible. Also, any attempt at getting even a partially controlled environment is invasive for the customer: They need to spend time helping with experiment setup, and/or limit system access to control workloads. The “controlled study” approach is not very attractive from a customer’s point of view. It usually takes quite a long time to execute, as well.

In modern distributed systems, a lot of operational data is collected. With the right tools, you can look at all that data instead of examining individual log excerpts. Instead of a controlled experiment, we perform an “observational study” where the various performance- affecting factors are controlled during analysis, by fine-tuning filters and aggregation levels. Essentially, we assume that our data already contains information about requests that are similar to the ones you would want to run in an experiment. You just need to find them.

16 Dec 2016 » Using DTrace to measure mutex contention in Ruby

by Tom Van Eyck, Workday Dublin

Working on Ruby code that contains a sizable number of threads can benefit from a tool that highlights which particular mutexes are the most heavily contended. After all, this type of information can be worth its weight in gold when trying to diagnose why a threaded program is running slower than expected.

This is where DTrace enters the picture. DTrace is a tracing framework that allows for the instrumentation of application and system internals, thereby enabling the measurement and collection of previously inaccessible metrics. DTrace makes it possible to gather a truly ridiculous amount of fine-grained information about what’s happening on a computer.

DTrace is available on Solaris, OS X, and FreeBSD. There is some Linux support as well, but it might be preferable to use one of the Linux specific alternatives instead. Info about these can be found here and here. At the time of writing this, it was announced that DTrace will be making its way to Linux as well. Please note that all work for this particular post was done on a MacBook running OS X El Capitan (version 10.11).

15 Nov 2016 » Docker Global Mentor Week Meetup

Kicking off at 18:45 sharp this Thursday @ Workday Dublin we will be hosting a Docker meetup as part of the Global Mentor week. There will be a short presentation followed by breakout session / labs.

Docker Global Meetup

More details including installation guide for Docker and labs material here

Hope to see you there.

20 Oct 2016 » Functional Kats Meetup at Workday Dublin

Join us at Workday Dublin for the October Functional Kats Meetup. There will be a talk from Juan Manuel Serrano on Type classes in Scala. Juan is a busy man, CTO of Habla Computing, founder and manager of Madrid Scala Meetup group and an organiser of lambda.world. There will also be a lightning talk from Jeremie Jost and a talk on Javascript and Elm.

Wednesday October 26th 2016 - 7:00pm @ Workday Dublin

All welcome. Hope to see you there!

Want even more FP? Functional Kats will be holding Kats Conf 2 on the 18th February 2017. This is a single-track conference - so no FOMO. Tickets will go on sale next week.

18 Oct 2016 » Calling All Superheroes to the Workday Booth at Grace Hopper

Look it’s a bird, it’s a plane, it’s Workday at Grace Hopper next week!

We will be at the world’s largest gathering of women technologists in Booth #1305 to:

chat with folks
give away schwag
schedule interviews
help us raise $$ for Capes4Heroes by:
1. picking up your Workday superhero cape at our booth
2. taking a pic of yourself wearing the cape at GHC
3. tagging the photo #GHC16 & #Capes4heroes on social media

For each tagged photo, Workday will donate to Capes4Heroes!

Check our latest video about Workday culture to see how much fun we’re having changing the way people work.

Hope to see you at Grace Hopper. And we’re up, up, and away!

17 Oct 2016 » Deep Learning in Production Meetup

In August 2016, Workday hosted a meetup in the San Francisco office on the topic of Deep Learning in Production. Deep learning has proven to be effective in solving numerous machine learning problems, and found a wide adoption in many applications from computer vision and speech recognition to anomaly detection. Despite the simplicity of deep learning algorithms, successfully and effectively deploying trained models to production is still at large an under covered topic.

In the first video Jim Stratton, VP of Software Engineering at Workday, gives an introduction of Workday’s machine learning and deep learning initiatives. Then David Kale of Skymind presents a walk through of DL4J, one of the promising deep learning frameworks that’s been gaining traction in production environments.

In the second video, Leo Tam of NVIDIA previewed DGX-1, the world’s first purpose-built system for deep learning.

This tech talk was co-organized by Skymind and Workday.

23 Jun 2016 » What is Serverless?

There has been a lot of buzz lately in the online tech world about a new type of architecture coined “serverless”. There are likely many definitions of what this architecture really means and I have found it to be an exciting development.

Traditionally, when people sit down to draw out a web application, often the first decision is which web server to use. In the Java world, Apache Tomcat would be a likely choice. You would download it, install it, configure it, build your application, and then deploy to it. You would typically do all of this on your local system. Now say you want to have a staging environment or deploy to production instances, the same steps need to be done. Likely firewalls have to be configured and servers need to be purchased, and then maintained and upgraded. The full monty server infrastructure that we are familiar with. Additionally, say your application load now extends beyond a single server. You need a load balancer and you need to figure out how to spread the load across your servers and scale them out. Oh wait, let’s not forget we need a backend store! Where do I download MYSQL? How do I install it? How many database servers will I need? What about replication? Too many things to keep track of so I’m just going to throw these concerns over the fence to Infrastructure or Ops.

But say you were to build this application today. Would you do things differently? You probably would.

For starters you would probably containerize your application, which abstracts away the infrastructure details and promotes virtualization and automated deployments. This is great. But you are still dealing with the abstraction of a server. But as long as you are careful and keep them stateless, scaling them out shouldn’t be difficult. The infrastructure will likely support auto-scaling containerized applications based on load. But as you add features and functionality to your server code, it continues to grow. It becomes fat. The code base becomes large and complicated. Perhaps we now need to think of a way to break up the server into many servers.

What if you could build an application that abstracted away the server layer? I would give it some serious consideration.

Recently, cloud platform providers like Google, Microsoft, and Amazon have begun to provide ways to build applications that are not server-centric but instead “server-less”. Rather than composing your application as servers you compose your application as stateless functions. For example, you can have a function that services your HTTP requests, a function that is triggered by changes to your persisted data, a function that is triggered on a schedule, a function that is invoked by other functions, or a function that subscribes to a message topic. Each function is independent of other functions and can be deployed and updated in isolation. Each function can also scale independently. There is no function startup phase, once It’s deployed it becomes ready to receive requests. Startup doesn’t make sense because there is no server state to initialize. As you can see, this model is very different from a server based model. It is distributed, scalable, and stateless at the core. Combine these serverless functions with other “serverless” components like a serverless datastore, then you truly can develop an application without having to think about servers.

It is yet to be determined if serverless is just another fad or will it be something that will stick around. I can see it’s applicability when it comes to encapsulating short units of work like processing an image or http request logic, but it needs to be proven whether one can successfully compose a full enterprise scale application. Personally I think you can, because given that a function is a primitive construct then you can combine primitives to make more complex constructs. I think it’s just whether or not the tooling will mature and if the community will embrace it.

To finish, my definition of serverless is the ability to write an application above the abstraction level of a server, deploy it and have it automatically scale without managing the underlying infrastructure. I find that idea very appealing.

07 Jun 2016 » June Meetups at Workday Dublin

Meetups are coming in thick and fast in June. We have four this month all hosted at Workday Dublin.

June 8th Python Meetup

3 talks

Introduction to Docker for Python Devs
Plotly for Python
CPython Literacy.

Sign-up for Python Meetup here

June 21st Cassandra Meetup

User Defined Functions and Materialized Views in Cassandra 3.0

Sign-up for Cassandra Meetup here

June 23rd Project Management Institute

Portfolio Management in an Agile Environment

Presentation and open discussion on software portfolio management in agile environments.

Event Page for PMI Meeting

June 30th Functional Kats Meetup

Erlang Workshop

You’ve heard about let it crash, well how about you to to write this. The kats are crashing systems. What could possibly go wrong?

Sign-up for Kats Meetup

All talks start at 6:30pm.

Space is limited so click on the relevant links to sign-up.

Hope to see you there!

15 Mar 2016 » Dublin Scala Meetup with Bill Venners

Last week, Workday Dublin hosted a joint meetup for Scala Users Group and Functional Kats in our office in Dublin. We were delighted to have Bill Venners talking about asynchronous testing support in the upcoming release of Scala Test 3.0

With 3.0, ScalaTest will support testing in Scala.js, providing most of the functionality available to Scala developers. To properly support Scala.js, ScalaTest has been extended to include asynchronous testing styles which should make it much easier for developers to test their own async code in a non-blocking manner. For more information checkout Bill’s blog post about the preview release.

Keeping the testing theme going, Workday’s own Andrea Magnorsky gave a demo of property testing in F#. “You should write tests to defeat your worst enemy and your worst enemy is you!”

Conrad O’Dea @cjodea is an dev manager at Workday working with the Query Fabric team. The team are using in-memory data fabrics to build a scalable, resilient distributed datastore.

01 Feb 2016 » Ruby Meetup @ Workday

Finding Concurrency Problems in Core Ruby Libraries

Next Dublin Ruby Meetup February 16, 2016 will be held in Workday’s Office in Dublin.

Doors open 6pm. Food and Drinks. All welcome.

These events fill up quickly so sign-up here: [Ruby Meetup Link] (http://www.meetup.com/rubyireland/events/228342813/?rv=ea1&_af=event&_af_eid=228342813&https=off)

03 Dec 2015 » GOTOCon Berlin 2015 - Improving Correctness with Types

Iain Hull will be presenting at GOTOCon Berlin 2015

Abstract

This talk is aimed at Scala developers with a background in object oriented programming who want to learn new ways to use types to improve the correctness of their code. It introduces the topic in a practical fashion, concentrating on the “easy wins” developers can apply to their code today.

View slides online

References

Inspiration

The following links inspired this talk:

28 Oct 2015 » Julien Sirocchi's Talk at Workday Dublin

At a recent Scala Meetup in Dublin, Workday’s Julien Sirocchi gave a talk on “Algebraic Data Types and the power of Scala Type Classes”. This talk complements Miles Sabin’s talk (also published here) by explaining how Type Classes and ADTs can be combined to achieve ad-hoc polymorphism.

The video is now available here.

04 Sep 2015 » Miles Sabin's Talk at Workday Dublin

Thanks to everyone who came to the Dublin Scala Meetup at Workday’s new Dublin office. Miles Sabin gave a fascinating talk on “Codecs for free! - shapeless type class derivation in action”

For those who couldn’t make it, the video is now available here.

Enjoy!

07 Jun 2015 » Scala Days 2015 - Functional Programming with Arrows

Yuriy Polyulya presented at Scala Days Amsterdam 2015

Abstract

If you’re using a structure that’s very like a monad, but isn’t one, then maybe it’s an arrow. Arrows are a general functional programming technique, popular in Haskell, but so far undeservedly overlooked in Scala.

This is a short talk and study case about:

functions and syntax extension for arrows
arrow type-class
what are arrows and how to use them
arrows and categories
arrows and monads.

View slides online

26 Mar 2015 » Scala and Spark at Workday

Scala is a big deal here at Workday. Like many companies we have a large Java codebase developed over many years. Scala’s ability to seamlessly interoperate with Java code makes it easy to use Scala to build new functionality that interfaces with an existing codebase. Scala’s performance, rich type system, and emphasis on functional abstractions make it particularly well suited to big data applications.

The Workday Syman team focuses on building products to help customers make sense of all the data that they have stored in the Workday system. The name Syman comes from the data normalization technology that some of us built at our startup, Identified which was acquired by Workday in 2014. There were two previous iterations of the data pipeline; the first was written in Java, and the second in Scala. Neither of these were nearly as efficient or elegant as the one we have now, and that is largely due to another core technology, Spark.

Hacking away on the data pipeline

Spark is one of the hottest technologies in big data today. Compared to Hadoop Map-Reduce, and it’s associated libraries, Spark is quite young, but in our year of using it, we have found it to be quite stable and mature. Part of this is due to the fact that Spark’s RDD is very close to Scala’s Collection API. Mirroring Scala’s Collections API is part of what makes Spark so easy to use, even without getting to some of Spark’s higher level APIs like Spark SQL and the ML pipelines API. I’ve been quite impressed at the rate of development that the Spark core team has maintained. Even very new features have been quite stable for us so far, and a couple of them are becoming core to our products.

One of the new Spark APIs that we have started using to great success is Spark Streaming. Like many streaming applications, both batch and streaming data processing are extremely important for our search indexing. This is part of what makes Spark Streaming well suited to search indexing. As data is updated, the streaming job takes care of keeping indexes up to date with the latest data. When we want to make improvements to how we generate documents, we can run a batch job that reindexes everything. Critically with Spark Streaming, the API is similar to the Spark API, so code is similar. It’s only at the top level where changes are necessary.

If you couldn’t tell already, we are huge fans of Spark. If you have any interest in big data we would strongly encourage you to check it out. We got really lucky with the timing of starting a lot of these projects (right around the time that Spark 1.0 was released). Being able to build things from the ground up in Spark without having to port an existing Hadoop pipeline has been pretty cool and really exciting. If you’re the kind of person that also gets excited about the idea of building production applications with Spark, you should check us out!

About Andrew

Andrew Bullen is a Senior Software engineer on the Syman team at Workday. The Syman team uses Scala and Spark to help Workday’s customers make sense of massive amounts of data.

17 Mar 2015 » Scala Days 2015 - Improving Correctness with Types

Iain Hull will be presenting at Scala Days SF 2015

Abstract

View slides online

References

Inspiration

The following links inspired this talk:

05 Feb 2015 » Scala Typesafe Wrappers

Why Wrap?

Primitives like Strings, Ints, and Booleans are excellent at holding values, but in most use cases they are weakly typed. Primitives do not carry any semantics or invariants. For example, a string can store an email address but a string is not an email address. If you store an email address in a simple String, any invariants such as ‘is it valid?’, have to be assumed or checked every time the value is used. This encourages bugs at worst or bloated and overly defensive code at best.

So why are primitives so over used?

They already exist and are easy to use.
The cost of using primitives is not paid by original developer who knows where is safe to assume a value is valid and when it must be checked. However people who maintain or use this code are not so lucky, they have to ensure all preconditions are met before calling the code and don’t know when to assume or check any postconditions it guarantees.

For a more complete explanation on the ills of primitive typing see this excellent blog The abject failure of weak typing.

We can strengthen primitive types by wrapping them with a simple type to encode our semantics and invariants. We verify the invariants only once when we create the value and the compiler proves they are true everywhere we use it.

class WorkerPool(
  initialSize: PositiveInt = defaultInitialSize,
  maxAllocation: StrictPercentage = defaultMaxAllocation) {
    // ...
  }

For example the WorkerPool class uses typesafe wrappers to ensure the values passed into the constructor can be used without any extra validation. Here the compiler ensures that initialSize is never negative and that maxAllocation is a percentage value between 0 and 100. This simplifies the implementation as WorkerPool does not need to validate the values. It also makes the class easier to use as you know exactly what values it supports.

13 Jan 2015 » OpenStack Election: Request for Support

This week the OpenStack community will elect eight members to be part of the 2015 Board of Directors. For more information see 2015 Individual Director Election

Workday is participating this year by proposing Edgar Magana as candidate for the Board as Individual Director.

Edgar has been an OpenStack member since 2011 as a part of the founding team of the networking project Neutron (formerly known as Quantum). He is still a core developer and his contributions have been extended to projects such as nova, devstack, docs and heat.

Edgar has a PhD on Distributed Systems from the Universitat Politècnica de Catalunya, Barcelona Spain. Edgar has experience in Cloud Computing, Network Virtualization, Software-Defined Networking (SDN), Network Functions Virtualization (NFV), OpenFlow and most importantly OpenStack.

While working in companies such as Cisco, and PLUMgrid, Edgar has completed successful OpenStack deployments in heterogeneous and scalable infrastructures using vendor-specific and commodity hardware. He has contributed back to the community improvements to both OpenStack code as well as configuration systems such as Puppet and Chef. Edgar has promoted the growth of the OpenStack community across continents, giving presentations in conferences in Asia, Europe and America.

We wish Edgar well in his commitment to make OpenStack a truly scalable, operable and production ready Cloud Management System and we would ask to have your support for him in this year’s Election.

08 Dec 2014 » Scala eXchange 2014 - All you don't need to know about typeclasses

Julien Sirocchi will be presenting at Scala eXchange December 2014

Abstract

Typeclasses provide a means of adding behaviour to existing types retroactively (known as ad-hoc polymorphism). They can make libraries easier to use, more powerful and more generic. This presentation is a distilled narrative from the long journey discovering how to effectively use typeclasses in Scala.

View slides online

References

About Julien

Julien Sirocchi @jsirocchi is a Senior Software Developer @ Workday and is a member of the Grid Team. The Grid Team use Scala, Akka, Play and Spray to provide Workday’s elastic job execution evironment.

22 Sep 2014 » Scala By The Bay Videos Online

Videos are now online including presentations from Workday’s Bjorn Hamel and Phil Monroe

15 May 2014 » Managing a Job Grid using Akka

The Job Grid

Every month Workday’s Integration Cloud Platform (ICP) executes millions of jobs for its hundreds of customers: integration runs, PDF document generations, Payroll calculations and other types of background processing tasks. A feature of the ICP is that it allows customers and their partners to deploy code to Workday, which Workday then runs on their behalf. To achieve this, Workday has deployed a Grid of servers that can be dynamically allocated to tenants based on real-time demand. The Grid provides a scalable, secure, and isolated execution environment for all tenants.

Grid

The Grid is a High Availability (HA) clustered application that manages multiple pools of compute resources (Slaves). Different pools have different properties, for example, the size of compute resources, reservation policies, and so on, allowing different styles of background jobs to be handled. Within each pool, the Grid manages:

Dynamic allocation of Slaves to tenants based on demand
Queuing and prioritization of jobs
Supervision of all Slave servers and the jobs they execute
Handling the state transitions for all jobs

13 May 2014 » Scala: The Simple Parts with Martin Odersky

Scala Meetup San Francisco - May 22nd

Join Typesafe and Workday for a SF Scala meetup with featured guest, Martin Odersky, creator of the Scala language and Chairman & Co-Founder at Typesafe. Workday’s Aneel Bhusri, Co-CEO, and David Clarke, VP of Technology Services, will present how Scala is used at Workday, and then hand things over to Martin for a talk on Scala - The Simple Parts. Martin will present the simple principles underlying the Scala language and discuss the tradeoffs between generality, interoperability, and ease of use in its design.

Link to Meetup

10 Apr 2014 » Scala Meetup San Francisco April

Workday will be hosting a Scala Meetup at the SF office this Friday, April 11th.

Title: Reactive All the Way Down, Part 2 of 2

Description:

The world is going Reactive but not just for the back-end; UIs are also becoming Reactive. In this session we will walk through how to build an end-to-end Reactive application with Scala, Play Framework, Akka, and AngularJS.

James builds upon the previous day’s SBT-Web-fu and live codes a Play 2.3 app!

Agenda:

6:00 doors open
6:30 announcements, including 1-minute hiring pitches – prepare yours!
6:40 talks begin

Meetup Link:

05 Mar 2014 » Workday Sponsors Scala Days 2014

Workday are proud to be a sponsor of Scala Days in Berlin 2014.

Scala Days is the premier Scala conference and this year it’s back in Europe. The schedule is out already and it looks pretty great this year. The main problem is choice. We are already facing the tyranny of having to choose between Concurrency vs Big Data and Akka programming vs Reactive talks.

Somehow we’ll manage … we mentioned it was in Berlin right?

Hope to see you there!

09 Jan 2014 » Scala For Beginners

In Workday we’ve been using Scala for a couple of years now. The availability of engineers with Scala experience is getting better: starting from zero the only way is up! Most of the time we try to help experienced Java developers become Scala developers. Here’s how we approach the problem:

Welcome to a better world

The first thing we do is get developers excited about why we like Scala so much. There are many reasons:

Succint code / Less boilerplate - less code means less maintenance.
It’s Java-like and supports OO - so the transition can be gradual.
Powerful type sytem - let the compiler help you write statically correct code.
Collections API - aka no more nested for loops for you. Map, filter and reduce like a pro.
Closures - Java 8 will have some support, but you get that and more right now in Scala.
Functional Programming (FP) - there’s more to FP than just collections.
DSL’s support - create internal DSL’s and use them directly in your code.
Concurrency - Scala’s Futures make Java Future’s look pre-historic.

While these are all good reasons, we’ve noticed a tangible change in developers when they start down the Scala path that can be summarised as:

Learn to love your code (again)

05 Nov 2013 » Principles of Reactive Programming Course

Martin Oderskys new Principles of Reactive Programming online course from Coursera started this week. Last year a number of us really enjoyed doing the Functional Programming Principles in Scala. We use Scala and Akka a fair bit here, so we are really looking forward to this course.

If you are interested there’s still time to sign up for the course, the first videos were released this week. You need to know Scala and ideally have completed the previous course.

If you are in Dublin, Ireland - we are planning a study group at the Dublin Scala Meetup - come along and join us - all welcome.

01 Nov 2013 » Scala Exchange London 2013

We are going to the Scala Exchange in London - December 2nd and 3rd.

Looking forward to meeting-up with Scala experts and enthusiasts. If you are there say “Hi”.

22 Oct 2013 » RestTest : Exploring DSL Design in Scala

Iain Hull presented his take on Scala DSL design at the October Dublin Scala Meetup. This talk brought people through the process of designing a DSL to test REST-based web-services. It was a great talk and for those of you that couldn’t make it or wanted more information - here it is:

Developer Blog Compiling our endeavours

Recent Posts

09 Nov 2017 » We've Moved

19 Oct 2017 » Workday Platform Engineering Open House

05 Sep 2017 » Android — Inbox Material Transitions for RecyclerView

14 Aug 2017 » Ruby Concurrency: In Praise of Condition Variables

03 Aug 2017 » ShipItCon Dublin

25 May 2017 » Workday Hackathon

23 Jan 2017 » Drinking Your Own Champagne

Performance Debugging as Exploratory Visual Analysis

16 Dec 2016 » Using DTrace to measure mutex contention in Ruby

15 Nov 2016 » Docker Global Mentor Week Meetup

20 Oct 2016 » Functional Kats Meetup at Workday Dublin

18 Oct 2016 » Calling All Superheroes to the Workday Booth at Grace Hopper

17 Oct 2016 » Deep Learning in Production Meetup

23 Jun 2016 » What is Serverless?

Further reading

07 Jun 2016 » June Meetups at Workday Dublin

June 8th Python Meetup

June 21st Cassandra Meetup

June 23rd Project Management Institute

June 30th Functional Kats Meetup

All talks start at 6:30pm.

15 Mar 2016 » Dublin Scala Meetup with Bill Venners

01 Feb 2016 » Ruby Meetup @ Workday

Finding Concurrency Problems in Core Ruby Libraries

03 Dec 2015 » GOTOCon Berlin 2015 - Improving Correctness with Types

Abstract

References

Inspiration

28 Oct 2015 » Julien Sirocchi's Talk at Workday Dublin

04 Sep 2015 » Miles Sabin's Talk at Workday Dublin

07 Jun 2015 » Scala Days 2015 - Functional Programming with Arrows

Abstract

26 Mar 2015 » Scala and Spark at Workday

About Andrew

17 Mar 2015 » Scala Days 2015 - Improving Correctness with Types

Abstract

References

Inspiration

05 Feb 2015 » Scala Typesafe Wrappers

Why Wrap?

13 Jan 2015 » OpenStack Election: Request for Support

08 Dec 2014 » Scala eXchange 2014 - All you don't need to know about typeclasses

Abstract

References

About Julien

22 Sep 2014 » Scala By The Bay Videos Online

15 May 2014 » Managing a Job Grid using Akka

The Job Grid

13 May 2014 » Scala: The Simple Parts with Martin Odersky

10 Apr 2014 » Scala Meetup San Francisco April

05 Mar 2014 » Workday Sponsors Scala Days 2014

09 Jan 2014 » Scala For Beginners

Welcome to a better world

05 Nov 2013 » Principles of Reactive Programming Course

01 Nov 2013 » Scala Exchange London 2013

22 Oct 2013 » RestTest : Exploring DSL Design in Scala