How I became a Data Scientist

BOLOR TURMUNKH Photo

by Bolor Turmunkh, PhD, Data Scientist at Uptake Technologies Inc., Chicago, IL

First Steps

At the beginning of my fifth year of graduate school at the University of Illinois, with thoughts of impending graduation, I started thinking for perhaps the first time in my life about who I wanted to be. I had lived happily as an information hermit for four years. I had spared little thought for anything other than academic research. It would have been handy if I had kept up with career trends, sought-after skills, or internship opportunities. But as they say, the secret of getting ahead is getting started. So, I buckled down and got started.

After a quick google search on trending careers of the future and cross-referencing the required skills with my own past experiences, I landed naturally on data science. In this post, I will recount the path to my current position as a data scientist, and describe some differences between academic research and industry work – so that if you are considering the same options, you might be better informed about the trade-offs.

What is Data Science?

A famous Venn diagram (google “data science Venn diagram”) defines data scientists as having skills at the intersection of coding, statistics, and domain expertise. They are the people who take a business problem, go prospecting for available and attainable data, re-formulate the question in technical terms, design and implement a statistical and machine learning task, and re-interpret the results for the business client to ultimately answer the original question. That makes it sound like to be a data scientist you need to be a statistician and a computer scientist with years of industry specific experience. That’s not quite true.

The reality is, data science is both vast and new, with specializations and sub-fields quickly developing. Highly sought-after data scientists are people who are broadly familiar with all aspects of data science while being experts in one or two fields. It is a highly achievable career for mathematics graduate students – with some preparation.

How did I become a Data Scientist?

There are plenty of resources online that outline possible paths to becoming a data scientist. I will simply describe my own experience.

From the moment I realized I would enjoy being a Data Scientist to finding my first internship, I spent 9 months devouring online and free courses on Machine Learning and Python, sent out dozens of applications, got two interviews, failed miserably at one of them and lucked out with the remaining employer, who was willing to give me a chance. It was a small start-up in San Francisco developing enterprise software in Natural Language Processing. They posted internship positions on their website alone, and I daringly emailed the founder directly with my best pitch about why I would be a good match. I say daringly because it is not standard practice at larger and more established companies. But anything goes at start-ups, and this one happened to go swimmingly for me.

During the three month internship, I learned intensely. My technical knowledge deficit was overwhelming at times. But here, my academic training was an asset. Living with overwhelming stress without it paralyzing you is arguably what “PhD grit” is all about. A harder adjustment was the social aspect of the workplace. Like any profession, there are jargons and topics, popular and unpopular opinions, the latest and meanest blog posts, all exchanged electronically in an open, yet entirely silent office. But I found mentors and allies who helped me feel at home.

This internship was just the beginning of my journey to becoming a data scientist. It took another two years and one failed job search cycle before I landed my current position.

How is being a Data Scientist different from being an academic?

You are no longer alone.

Intellectual isolation was the hardest part of my academic research experience. Apart from conferences in my particular field in mathematics, and research meetings with my advisor, I had no peers with whom to engage in frequent and technical discussions of the details of my work. That is no longer the case in industry. Not only are my coworkers ready to get as nitty-gritty into my project as I wish to go, they also possess a wealth of experience dealing with similar projects and are happy to share their expertise. Learning in such an environment is exponentially faster than learning alone.

Project time frames are shorter.

Time frames differ significantly from company to company. Larger companies tend to tackle longer term projects. Software teams typically have shorter time frames due to the nature of the work. So, the scope and the strategic importance of  your day-to-day activities will vary depending on where you work. For me, deadlines for large projects are on a quarterly basis and smaller ones are weekly. The shorter deadlines are oftentimes helpful since they force you and your managers to clearly define goals and criteria for success. On the other hand, short-term goals can sometimes feel short-sighted, if your team’s priorities change drastically.

You won’t always get to decide what to work on.

This one is a spectrum. Companies such as Apple prefer to set strategic directions and product vision from the top and have them permeate downward. More bottom-up companies such as Facebook prefer a more entrepreneurial feel. Most companies lie somewhere in between, which means you are somewhat in charge of what you get to work on. My team establishes quarterly priorities and project proposals together, which then go through a review process to make sure the proposals align with company goals.

You have more resources.

As a graduate student, the main resource I had was my own time. As such, I was used to solving all my problems on my own. But as a team member, your goal is to arrive at a good solution in the most efficient manner possible. Doing everything yourself is not the most efficient way. Getting help is not only highly recommended, but expected of you.

Done is better than perfect.

It is an entirely new skill for most academics to weigh the costs and benefits of doing the job perfectly versus doing it fast. In industry, one makes this trade-off every day.

Closing thoughts

The qualifications and projects of a data scientist are quite different from those of an academic mathematician, and yet the actual work is quite similar in nature. The great majority of a data scientist’s time is spent defining and re-defining an ambiguous problem until it can be clearly stated, and then solved.

Once a data scientist finds interesting results, it is crucial to communicate them to the end customer. Building a story around a complex issue, supporting that story with evidence derived from data, and interpreting the results into a concrete recommendation for the customer, are the central tasks of the job. From this perspective, your graduate training in mathematics, statistics or operations research will provide a strong foundation for moving into data science.

Good luck with your career transition and job search!

What I know now that I wish I had known then!

Kristine_Jones

Kristine Jones, PhD – Senior Data and Applied Scientist, Microsoft
Opinions expressed are my own and not necessarily those of my employer or any institution that I may be affiliated with.

When I was first approached about writing this post, I was asked to try to convey what I know now that I wish I had known as I was applying for jobs out of grad school.  My response to questions such as this is always that there is no one course I wish I had taken, no one skill I wish I had acquired, no one opportunity that would have pushed my early career down a dramatically different path. This is not to say that I haven’t reaped the benefits of broad exposure to numerous skills commonly bullet-pointed on tech industry data scientist job descriptions. I would not be doing my due diligence if I pretended otherwise. That being said, hiring decisions for mathematicians based exclusively, or even primarily, on those bullet points are poorly considered. (Learn some coding, stats, and optimization methods, though … it can only help you).

Ultimately, the most valuable skill that a mathematician brings to any team is her ability to abstract the core technical problems the team is facing and provide the basis for solving these problems across all of the scenarios in which they assert themselves. The crux on which this skill rests is an education that teaches students to think deeply and critically about mathematics, both independent of and relative to the breadth of contexts in which it appears.  What follows is a personal history of how I arrived at this view.

From my vantage point as an undergraduate, the University of Chicago embodied this notion of linking independently considered critical thought with the scope of its applications.

The opposite criticism is often levied against the University of Chicago: that it is overly devoted to abstraction in place of the concrete. Perhaps the most telling supporting evidence of this criticism is the slogan “That’s all well and good in practice, but how does it work in theory?”, text often displayed along with the University’s logo on t-shirts sold by student organizations. Other examples abound, even (perhaps especially) from within the mathematics curriculum.  As a senior studying abroad at the University of Chicago Paris Center, I took a course in representation theory.  On the first day, the instructor, trying to assess the background of his students, asked the class who could define a vector space.  Several hands shot up, mine among them, all with the same answer ready – a vector space is a module over a field. The basis for the criticism is not unfounded.

But, while the University does certainly value abstraction, it does not do so without purpose. Even their motto declares this intention: Crescat scientia; vita excolatur.  Let knowledge grow from more to more; and so be human life enriched.

The goal of all the theory, of all the abstraction, of all the critical thought, is to enrich human life.  To make meaningful changes in the way scenarios play out and issues are addressed. To enhance the ways people think and people act.

This idealistic motto filtered down to the mathematics undergraduate curriculum in two ways that were evident to me.

First, mathematical abstraction was not left to the math majors. Proofs and notions of generalizability were at the core of even the most introductory of mathematics coursework, both calculus and non-calculus based. To these students especially, the value to be found in the mathematics coursework was not in a particular formula or tool that they may or may not ever need, nor in seeing jargon from their chosen field strewn out across rote problems.  Instead, they left their math classes with the ability to reason deeply and strongly about quantitative questions they would encounter in the future.

Second, if you wanted a B.S. in mathematics instead of a B.A., you had to take three non-introductory courses in a related minor field. Physics, chemistry, and computer science were common choices, in addition to a more structured option in economics, which was my selection.  This was not for an applied math degree, where the minor field requirement was even larger, but for a degree in pure mathematics. Amidst the modules over the fields, you had to understand what was real about the math you were studying.

I left the University of Chicago not only with lofty examples of abstract concepts and their footprints across mathematical theory, but also with an ability to “suss” out broad connections to this theory across all manner of problems I encountered. I headed off to graduate school to pursue a passion for what I thought was mathematics, with no particular career path in mind. I saw, and still see, a graduate education as an end in itself.

Let’s be honest here, though. Grad school is not for everyone. Grad school is hard.

That’s ok, it’s supposed to be. At its very best, graduate school asks that the student be lost in a sea of seemingly unconnected examples, looking for problems and finding solutions, ultimately weaving everything together into a single theoretical fabric.  It is from this process that graduate education derives its worth. Completing this work is an incredible demand on any person. I was lucky to have a truly amazing graduate advisor. He excelled at guiding his students from examples to theories and back again, all the while allowing them to find their own way.

Which is not to say I was always successful in seeing that path.  I spent one summer doing hundreds of matrix computations only to reach the conclusion that I would need to do thousands more to see if there was any pattern that might indicate the presence of an underlying theorem. My advisor’s comment upon hearing this was that I needed to come up with an “exit strategy for the project,” without saying much in the way of how I might do that. That was the “finding my own way” part.

Moving forward out of that moment was one of the hardest things I have ever done.  It was only when that work was nearly completed that I saw the value of my education was in the struggle to see connections between specificity and generality, regardless of any career decision I would make. More than that, I realized that my passion for math was really a passion for executing this skill. This insight came at a great moment because I was about to graduate, and seemingly out of nowhere, previously unconsidered options abounded.

I sent out resumes and went on interviews.  Microsoft stuck.  Since I’ve been here, I’ve designed large-scale machine learning systems, implemented component-scale algorithms, coordinated projects across research and engineering teams, and performed executive-facing analyses in high-value areas. Very little of this has much relation to the content my thesis (although I did run a lecture series on the theoretical underpinnings of homomorphic encryption that one time), but I doubt I would have been able to accomplish much if I hadn’t gone through the process of writing it. All of my solutions for Microsoft draw their impact from seeing connections between many smaller problems, with many commonalities, and solving them all at once – quite similar to academic mathematics in form, if not in function.

ICME Data Science Workshop

ICME will hold one-day summer workshops on Fundamentals of Data Science from August 14-18th at Stanford.  You can sign up for one workshop, or several, with topics ranging from Machine Learning to Natural Language Processing to Programming in R. Visit our Summer Workshops website for more information and to register.  

Feel free to spread the word! We hope to see you there.

Judy and the ICME team

Judy Logan

Institute for Computational and Mathematical Engineering (ICME)

Stanford University

Women in Data Science (WiDS) Conference

Threefold advice: making the jump from geometric group theorist to computer vision specialist

Lucas_Sabalka

by Lucas Sabalka

I began my mathematical career as a research mathematician, but now I work in industry even though my degree is not in an applied area. With so few academic jobs available recently, transitioning to industry is becoming more common for mathematics PhDs. So to help any mathematicians thinking about that transition, let me tell you how I got where I am.

I had always planned on being a professor as I pursued my PhD. That’s what I became: after two postdocs and a decent rate of publication, I got a tenure-track position at a research university. A career in academia has significant pluses, including the promise of tenure and thinking about interesting problems all day. However, through the course of these positions, I gradually realized the impact of two important minuses of a career in academia. One is that, with academic jobs so few and far between, you typically do not get to choose where you live. My wife and I are from Nebraska, and wanted to end up close to family and friends. The second is that research is driven by self-motivation. That’s good for someone like me who is highly self-motivated, but it can also add undue stress: I was easily on-track for tenure, but found myself pushing hard to make a name for myself with little recognition.

The experience that changed my career path from academia to industry was a consultantship. A co-author and good friend of mine, Dr. Josh Brown-Kramer, was working as an applied mathematician at a start-up tech company in my home town called Ocuvera. I have an undergraduate degree in math, computer science, and history, and together with Josh, I had competed in and won a few programming contests back in the day. I had done very little programming in the intervening years, but I had enough knowledge to pick up coding quickly. Josh put in a good word for me, and got me a full-time consulting position one summer. That position turned out to be a good opportunity for the company to see that I was a good fit culturally and could contribute positively to their products, as well as a good opportunity for me to see what working in industry was like. A few months after my consultantship ended, the company extended me a full-time offer. It was a difficult decision to make, but the draw of moving back home and (what was for me) the lower stress of working in industry led my decision. I took the plunge and switched careers: from “mathematician” to “applied mathematician”.

That transition was anxiety-inducing. I had prepared for many years to be in academia. It had the promise of tenure, and it was familiar. Industry was scary: what if my company folded? How would I handle the different stresses? In retrospect, I should have had more confidence in myself. I now trust that I will be able to find another job if my current job were to disappear. The stressors are different, but overall my stress levels have decreased. I have more time for hobbies, including advocacy and volunteerism (I speak with elected officials and thought leaders about climate change and the transition to a clean energy economy).

My job is Computer Vision Specialist. I develop algorithms for computers, equipped with 3-dimensional cameras, to automatically monitor patients in hospital settings. If the algorithms detect risky behavior from the patient that could increase their risk of falling, they automatically alert hospital personnel to determine an appropriate course of action. Falls cost hospitals and patients billions of dollars per year and can result in death. Helping reduce fall risk and introducing automated monitoring should reduce health care costs as well as improve patient outcomes and save lives. It is rewarding to feel like this project could help improve people’s lives.

My dissertation was in geometric group theory, a topic at the intersection of algebra and topology. While my job does not call for geometric group theory or really any graduate-level mathematics, I do use undergraduate-level mathematics concepts extensively, including statistics, probability, calculus, Euclidean geometry, various computer science algorithms, and linear algebra. We use machine-learned algorithms and we also write computer vision algorithms by hand. Consider, for example, taking an array of points in 3-space representing a single camera frame from a video stream of a hospital room, and trying to identify exactly those points that represent a bed. What properties of a bed are important, and how do you quantify that in a way a computer could evaluate? Once you know where the bed is, which points in 3-space represent the patient, and which the nurse? How will you deal with noisy or missing data? I may not be using the tools of my specialization, but I am using the problem-solving skills that I developed while pursuing my degree. My degree is not applied, but having a PhD in mathematics in any subject shows that you’re good at problem solving.

My advice to mathematics PhD students considering industry for work is threefold. First, remember that your degree will mean you are a very good problem solver, and have confidence that there are companies that value your skills. Second, it’s a good idea to get some classes under your belt that could help you in your desired fields: computer programming, statistics, probability, finance, or any classes that could apply in industry. These classes aren’t necessary, but can distinguish you from other candidates and help prepare you for the transition. Third, if possible, I recommend finding an internship in the field you’re looking at. This will give you valuable experience, help you know what to expect, show you whether you’d like that industry job, and will help you on the job market. Even if you don’t take other classes or have an internship, companies provide new employees training for their new roles.

If you are faced with a career change and decide to leave academia, remember: a PhD shows you are a good learner and you have the problem-solving skills necessary to succeed in industry!

Contact: sabalka@gmail.com

Academia trained me for a BIG career

 OLYMPUS DIGITAL CAMERA
by Peter D. Horn

I am honored to share some career advice with the young and mathematically-inclined. When I fit that description, I felt a lack of diversity in the opinions and advice I was hearing from my mentors. This wasn’t their fault, but mine. Classic case of selection bias, as I only sought advice from my professors.  My first recommendation is to connect with many math folks who have walked a variety of paths to get a sense of what is out there (reading the posts on this blog is a great first step!).

When I was finishing up my math major, I felt there was more math for me to learn, and I went on to get a PhD in low-dimensional topology. As a grad student, I was encouraged to pursue a postdoc. By the time I was deep into my postdoc, I had a tenure-track job in my sights. It wasn’t until my third year into a tenure-track position that I evaluated my career choice and realized I would be happier doing something else.

I reached out to a few friends from grad school who went into government and industry, as well as a couple former academics who transferred to tech and finance jobs.  I did a little research to see what was out there, and found “data science” to be a broad enough field to entertain my intellectual curiosities (e.g. machine learning algorithms) while providing plenty of job security (i.e. strong business demand).  Currently, I am a data scientist at the MITRE Corporation, a non-profit company that does R&D for many federal agencies.  I love working at MITRE because I get to define what type of data scientist I want to be.  In my first year, I worked on research projects involving machine learning and agent-based models to drive policy analysis, and I prototyped a web-based simulation tool to explore workforce strategies for the VA.  It’s great to be at a company where the work is challenging and impactful.

While in the transition to industry, I realized that much of my academic training and some of my hobbies positioned me to be an attractive candidate.  As a math major/PhD candidate/professor, I had accrued a ton of experience teaching myself complex, abstract concepts. Employers seek out job candidates who can demonstrate the ability to pick up new things quickly.  Working in help centers/recitations/lectures, I had accrued a ton of experience explaining deep, technical material to non-technical audiences.  Employers like to hire teachers because they can put you in front of customers or use you to mentor young staff.  As a mathematician, you have surely gained similar experience.  Find a way to brag about your superpowers!

You’re going to need programming skills.  In my journey, I was lucky to have learned to code.  In college, I learned a bit of Java in CS 101.  In grad school, the math department hired me by the hour to maintain their website.  I chose to write up my homework in LaTeX.  Frequently, I would need to do some computations in Mathematica, Maple, Matlab, or Sage.  As a postdoc, I got bored one summer and wrote a couple of card games in Objective-C.  For a research paper, I needed to diagonalize some matrices over a non-commutative base ring, and I wrote the code to do this from scratch in Python.  Before I had even heard of data science, I had ten programming/markup languages under my belt, and I put all of them on my resumé to show employers that I am comfortable writing code.  If you don’t have experience programming, I recommend you pick up Python. It’s a good general purpose language.  Pick a project and use Python to attack it (e.g. implement matrix multiplication from scratch).

The last piece of advice I have is to acquire domain knowledge and to network. The biggest hurdle I had in my journey was learning to communicate with potential employers.  I decided to take online courses in data analytics and machine learning, and these courses taught me what people in industry care about, how they talk, and what tools they use.  I also participated in some coding and data science competitions online.  Since I had a noticable lack of business experience, these competitions were something I could point to as proof that I could do data science.  I would also recommend attending meetups in your area. In my experience, meetup people are very friendly and helpful.

Transitioning out of academia was scary, but it has been one of my best decisions.  At first I was worried I wouldn’t be what employers were looking for, but I learned that many employers want to build companies with people from diverse backgrounds. Don’t worry about trying to fit the mold.  Reach out to friends, former classmates, and friends of friends, and you will find all the support you need.

Blogpost: Parsa Bakhtary

fb_live

It is humbling to address future and current mathematicians, but as a former algebraic geometer myself, I will do my best to share with you my story. I work as a data scientist, which the Harvard Business Review in 2012 dubbed “the sexiest job of the 21st century,” at Facebook, which has been ranked by Glassdoor as one of the best companies for which to work. The path that led me from an eager math student who despised applications to where I am today has been a strange one, but the lessons I learned in my undergraduate and graduate math classes have had a profound impact on my ability to analyze concrete problems in industry.

After earning a B.S. in mathematics at UC Davis, I took a year off in which I decided to pursue a graduate education in the same subject. Seven years later, I finally received my doctorate from Purdue University, having written a thesis in the subject of algebraic geometry, and I was eager to take the path which would lead me towards a professorship somewhere. Unfortunately, I was unable to find a post doc in my home country of the US, so I took a position in Saudi Arabia at King Fahd University of Petroleum & Minerals, teaching calculus to aspiring petroleum engineers and occasionally publishing a paper. After three years there, I missed California and returned unemployed in the summer of 2012.

I quickly realized the job market for math professors wasn’t promising at the time, so I started looking for industry positions that would be suitable for someone with my background. After extensive Googling, I realized “data scientist” sounded like something I could do. I taught myself some Python and SQL, practiced analyzing and visualizing publicly available data sets in R and Excel, then started applying. After six months of unemployment, I caught a break and was offered a position at a startup in Chicago. The rest, as they say, is history.

My job at Facebook is unique in its flexibility and often quite challenging, though perhaps not in the same way as algebraic geometry. I have worked on game ranking, platform ecosystem health, comment ranking, celebrity usage patterns on Instagram, and discussion of TV show content on Facebook. I was lucky to be the first data scientist on Facebook Live when it launched, and our team helped grow it into one of the biggest live-streaming platforms in the world. The problems I work to solve can either be very technical, involving complex modeling and simulation, or it can be investigatory, requiring me to search for an explanation of an unusual phenomenon, or it can even be exploratory, such as trying to answer vague questions like “What makes a mobile game fun?”

The analytical training that we mathematicians receive put us at a unique advantage in the field of data science. The rigor we’re accustomed to help us break down a general question into concrete analytical pieces which we can answer with data. It is easy for us to spot errors in thinking, or situations where the evidence doesn’t actually answer the question. After learning some basic statistics and the familiarity with an analytical data manipulation environment (e.g. R or Excel), any mathematician can rapidly become a data scientist. The field of data science is also vast, as one can focus on subfields such as product analytics, visualization, or machine learning.

The biggest misconception people have about data science is that they think we all know how to program and have spent many years writing code. While some familiarity with SQL and analytical software is often desired, we are not programmers. We are, if anything, the voice of evidence at a company. We are there to help shape our colleagues’ understanding and intuition based on the data that we see, and to give actionable recommendations that will improve existing products and help define the appropriate strategies. It’s a fun job, and a great option for all mathematicians interested in industry.

parsa_thumbsup

CC BY-NC-ND