Recruiting

#Hadoop Architecture and Software Development in #Boston – Massive Consumer Data Pipeline

Posted by Steve on July 13, 2015
Posted in: Recruiting. Leave a comment

HadoopInBoston2Boston-area analytics looking to bring on a few more devs to work on the various short and future projects – some of the details are below (any more detail requires a signed NDA or the possibility of death). Roles are senior to team lead – both have architecture elements; openings are because of life changes for folks who have moved on, for people who have been promoted, and for adding to the teams as a result of both new business and for customers requiring substantially more functionality.

The head of software technology is pretty darn easy-going and is a Luke Starwalker wannabe who keeps a light saber handy (but seems to always be in need of new batteries). The company began as a startup, grew, made money, yet retains some startup feel to it (just like adults have the knack for acting like goofy kids when needed). You don’t have to be in the office all the time and if you wear business casual you will likely be met with multiple Golden Retriever head-tilts.

So read on – and LMK what resonates with you (my email is at the bottom). No résumé is needed because we’re just talking…

Details About The Pipeline

  • Pipeline processing utilizes Torque (batch server) and Hadoop
  • Data needed for processing are stored in MySQL and Vertica databases
  • Vertica is used for generating reports used by Data Quality and Data Acquisition; pipeline jobs are orchestrated by a proprietary interface built on top of Luigi; ad-hoc access to pipeline files in HDFS is provided via Hive
  • Initial prep of pipeline files, such as cleansing and removal of sensitive data is via Python scripts and C++ compiled programs; the bulk of the rest of the processing is implemented as Python scripts and Map/Reduce jobs in Hadoop using streaming interface

The Company’s Development Approach

  • Agile and Scrum with 3 weeks Sprints with emphasis on high priority stories (typically 80+% acceptance of stories)
  • Rally project management
  • No special support bucket
  • Scrum and tickets for DevOps

The Software Development Flow

  • GIT for source control; repos are hosted internally
  • Gerrit for code reviews and merges (developer pushes changes in their topic branch up for review)
  • Jenkins for deployment and unit tests (kicks off master merge to Gerrit)
  • Pip install for Python packages, internal PiPy server, everything runs in virtualenv

The Software Development Teams

  • Data Technology – Data Delivery (Pipeline)
    1. Distributed processes, batch processing, Map/Reduce jobs, application of Data Science models for inference and weights generation
    2. Cloud Computing with Hadoop, Torque, MySQL, Vertica
    3. Python, bash, C++
  • Data Technology – Data Mining tools
    1. Development of the tools for Data Miners, mostly CLI
    2. Data Extraction using Hadoop, Torque, MySQL, Vertica
    3. Stronger emphasis on efficiencies and response times
    4. Data Aggregation and metrics generation, application of the weights
    5. Python, bash, Java
  • General Interfaces Team
    1. Interface layers on top of Hadoop, Torque, Vertica, Luigi
    2. Authentication and web services
    3. Application deployment
    4. Python, Java, network/system layers
  • Syndicated Products Team
    1. Python Django, Backbone, jQuery and various JavaScript charting libraries
    2. CSS3, HTML5, LESS
  • Data Collection Agent Team
    1. Browser extensions for IE, Chrome and Firefox, proxy for Windows
    2. JavaScript, C++, Windows Interfaces, Install, Python for reports generation and download software website
  • DevOps
    1. DevOps
      • Production runs of pipeline, backfill, re-runs
      • Pro data publish
      • Hardware troubleshooting
      • Web Servers tuning, Linux in production environment, Windows Servers, IIS, Apache, NFS, SQL, SSH/SCP/SFTP
    2. IT Labs: PC/Laptop support, HelpDesk Change MGMT

Future Tech Initiatives

  • Dynamic Normalization Methodology, Mobile
    1. Complete overhaul of Pipeline
    2. Complete overhaul of Data Mining tools
    3. Redesign of Custom Products: Moving to Dynamic Normalization, simplifying User Experience
    4. Segment Management System: Allow Data Miners to generate, maintain and query a set of standard data segments with automated daily data pulls
  • Next Generation Data Collection
    1. Local Proxy Windows Service
    2. Keeping up with Browser evolutions

As promised, here’s my email (just click it to ping me): levy.steve@gmail.com

Thanks. ~Steve

Recruiting Webinar: Hiring Techs is Easy…Right?

Posted by Steve on July 1, 2015
Posted in: Recruiting. Leave a comment

WebinarHiringTechs

Hardly. Grab the popcorn and listen to my recent presentation on RecruitingWebinars.com: “Hiring Tech Talent Is Easy…Right?”

Click here for the almost hour-long webinar; you can also download my slides from this page?

Any questions?

Hiring Techs is Easy, Right?

Posted by Steve on June 18, 2015
Posted in: Recruiting. Leave a comment

WebinarHiringTechsIt’s not that there’s a shortage of great technical talent – but there does seem to be a significant gap from the sourcing & recruiting professions understanding many things technology.

This webinar addressed 4 core areas around hiring technical people:

  1. The problems sourcers & recruiters have with sourcing, engaging, and recruiting people for technical roles
  2. Understanding the technical mind around work and careers
  3. Identifying specific ways to source, engage, and recruit these people
  4. Implementing better ways to assess the skills of these purple squirrels.
Click here for the entire webinar (about one hour) as well as a link to download the Powerpoint deck ( a new tab will open).
Thoughts?

Trees, Culture, and Recruiting

Posted by Steve on March 16, 2015
Posted in: Culture, Recruiting, Recruiting Association. Tagged: , , . 1 Comment

552380701_snl_church_lady_xlargeSounds as if another ubiquitous #CultureChat has taken place at SXSW replete with recycled lines about what culture is to the experts. Could they all be wrong?

While it’s relatively easy in anthropological terms to talk about how culture is passed on from generation to generation (or management team to management team, employee to employee, customer to customer) via things like storytelling or analysis of clay pot remnants, it’s the initial description of “What is culture?” that vexes almost everyone in HR and talent functions.

Almost everyone speaks of their culture in terms of its uniqueness, how special it is, how different it is from all others – using the same words as everyone else.

Well isn’t that special?

When groups of experts gather ’round to discuss the many shades of culture by using the same language how much learning actually takes place?

I think the reason most use the “standard culture word library” is because accurately describing culture is really, really difficult. It’s this “lapse” in analysis and erudition that leads to the typical “I’ll know it when I see it” action taken during recruiting.

“I’ll know it when I see it but I can’t really describe it.”

How frightening is this? Can you fathom how many great people and performers we’re tossing to the side – and yet at the same time claiming to others that we’re experts at recruiting?

I liken it to the saying, “If a tree falls in a forest and no one is around to hear it, does it make a sound?”; in reality, most everyone inside a company either experiences the culture of the company second hand or takes the word of someone else.

Does the tree make a sound? Of course it must. Is our culture special? Of course it is.

It has to be.

How many in recruiting – especially those newer to it – actually spend significant time inside the groups being supported, with enough time spent observing and collecting data like the great anthropologists? Yes, it’s a rhetorical question.

Instead, most all recruiters simply take the word of their charismatic leader/founder, and/or authentic marketing expert, and/or HR branding guru. Think about how long anthropologists study the remnants of a people who long ago perished before they begin to understand the culture. Yet here in HR and recruiting we take the word of an entrepreneur whose company has been in existence 12 to 18 months?

That’s one of the core problems in recruiting – the forest we think we see through the trees isn’t what we’re describing with our words nor actions. If this isn’t what’s real then I posit that the profession is comprised of lemmings and sheep who need to have their collective eyes and ears examined.

As harsh as the last sentence is, it it spotlights that we’re darn near in the middle of a downward spiraling vortex that’s sucking our collective recruiting reputations down the drain then I’ll deal with the fallout.  It’s time to plug the sink, hire better plumbers, fix the pipes, and enjoy the outcome.

Yes, the falling tree in the forest does make a sound and it’s time to prove it.

Starting now…

[cross-posted on RecruitingBlogs.com]

Spamming The LinkedIn Way

Posted by Steve on March 15, 2015
Posted in: LinkedIn, Recruiting. Tagged: , , . 4 Comments

Since there are so many ways that as recruiters we’re not getting an appropriate bang-for-the-buck from our LinkedIn investments (and more importantly, in any way aligned to the heights of their marketing bluster), I’m going to focus this week on the many ways that we are spammed via the LinkedIn messaging systems.

So to my frenemies in Trust & Safety, this blog’s for you!

For this week, I’ll focus on the Spam. You’ve received LinkedIn InMail that is unequivocally spamful. You scrunched up your face two sentences in and thought, “WTF?” You checked to see what level connection you have with this social Charlatan.

“*(^%$$%@#%#$, first degree.” You right click on their name and Open Link in New Tab…

You furiously scan up and down the profile of this person whom you let into your “trusted network” on LinkedIn. Your brow is beginning to sweat and your eyes are shrinking into beady little pools of molten lava that are ready to erupt. As you’re ready to shout out “DOUCHEBAG”, you click on Contact, because you want to call and skewer this bastard.

Only you see this:

opayq.com?

“Opayq.com is a domain used for Masked Emails for Abine’s MaskMe and PrivacySuite products. These are real user email addresses, created by users to protect their online privacy.”

Lovely.

Now Spammers are gaining control of LinkedIn accounts, adding their masked emails to your account, and sending your contacts LinkedIn InMails like this (I masked a few things myself to protect this person’s identity); notice the punctuation and grammar:

“I give you 100% guarantee of their legitimacy, a trial will convince you” – made me smile, because I was thinking of another kind of trial.

I hope Paul Rockwell has someone on his staff at LinkedIn who will forward him this post.

Fake Profiles, Dating Requests, Marketing InMails, Account Piracy – how soon before LinkedIn ends up as a running joke on TMZ?

And this is the ultimate tool for most recruiters?

[cross-posted to RecruitingBlogs.com]

Drinking The Sand

Posted by Steve on February 17, 2015
Posted in: Careers, Character, Human Resources, Leadership, Recruiting. 1 Comment

From The American President:

People want leadership, Mr. President, and in the absence of genuine leadership, they’ll listen to anyone who steps up to the microphone. They want leadership. They’re so thirsty for it they’ll crawl through the desert toward a mirage, and when they discover there’s no water, they’ll drink the sand. ~Lewis Rothschild

Lewis, we’ve had presidents who were beloved, who couldn’t find a coherent sentence with two hands and a flashlight. People don’t drink the sand because they’re thirsty. They drink the sand because they don’t know the difference. ~President Andrew Shepherd

From where I stand, this also applies to the Career & Job Search Counseling, HR, and Recruiting spaces.

Folks, we can do much better – for everyone’s sake.

Raise the bar (no, not the way Amazon does it ;)

A Recruiting Disruption

Posted by Steve on February 15, 2015
Posted in: Future, Recruiting, Training. 5 Comments

There’s a neat little discussion over on Quora about whether the recruiting profession is ripe for disruption. One of my favorite people – period – Glenn Gutmacher weighed in with a great comment about recruiting technology and you really should click over and read it. However, since not everyone has a Quora account, here are my comments to the audience (Glenn knows my feelings on this topic so I suspect he was smiling a bit while reading):

For all the entrepreneurs-to-be in the audience, the mere utterance of the work “disrupt” doesn’t imply that you’ll be able to come up with a sustainable model (oh wait, you don’t want to sustain – you want to build and flip your way into millionaire/billionaire status), primarily because you really don’t know what exactly is “broken” in recruiting.

All the folks who want to disrupt the heck out of recruiting – Uberize it if you want to get all hot and bothered – are assuming that the process is to blame, when in fact most of the process works fine until people get involved.

Glenn identified many a tool that presumably uncovers talent but as I demonstrated at a recruiting conference back in October, identifying is an ocean apart from engaging. Let me explain because there are two important parts of this.

Jobmatching has a tremendous amount of future utility but ONLY if specific skills, knowledge, and experiences can be matched to actual performance on the job. After all, the reason we hire people is to solve the problems that need solving, to develop the products that need developing, to service the customers who need servicing. Being able to find the “perfect match” – as the datingpreneurs like to call it – can’t be effective unless you have a frame of reference against which to measure how much of a perfect match it really was.

That frame of reference is called future performance (this is part one). Technology surely can help us here.

Can you develop a predictive model to get you part of the way there? Sure can. But there are many factors that weigh into the model and in the recruiting profession we’re just starting to collect the data.

Which points to a critical limitation in developing the model.

The recruiters (this is part two). Technology can only help so much here. This is the problem.

My talk at SourceCon back in October focused on HOW talent should be engaged by the profession. Many of the tools Glenn writes of can help produce great lists of people whose content is available somewhere online. I offer the word “online” because there’s a growing segment of people who are cloaking their online behavior, even eschewing online altogether for any number of reasons.

This is where a vocal segment of the profession has begun investigating what goes into becoming and retaining one’s stature as a “great recruiter.”

Personal experience aside, you need to understand that merely “disrupting” recruiting won’t change some of the statutory obstacles nor will it make average (or worse) recruiters anything more than average performers.

Disrupting won’t make your candidate experience great; people still have to be part of the equation. Disrupting won’t turn a bad hiring manager into Warren Buffett. The delicious part that everyone sees in the Tinder, Match, and eHarmony commercials is offset by what you don’t see – the mismatches, the ugly meetings, the divorces. Tools can help but tools don’t recruit – people do.

Glenn’s point about offline partnerships has tremendous possibilities. If you study the past “demise” of many industrial trades jobs like model making, welding (can’t print everything in 3D yet), woodworking, HVAC, etc., you’ll notice that we nearly “computerized” these to death. Rather than becoming extinct, a few companies are doing what he speaks of and creating their own corporate university around training people for these scarce jobs.

It’s really a question of creating content and being able to deliver them on platforms such as the one that’s been created by Kaltura. To this end, the talent model is “build from within.”

Very smart move.

What do you think would truly disrupt recruiting?