Performance Evaluation Strategies

Cross-posted from my other blog. Apparently, aspects of the performance tracking system I've developed might be useful to a wider audience than I'd anticipated!

Here's the text of an email I sent to a brand new (and very bright) graduate student recently. The student had asked me to "have a quick look at" a one page proposal for a small internal stipend competition.

---------------------
"Hi [name]

The bulk of the application is fine – I’ve attached an edited version with track changes on so you can see what I’ve done.

In addition to these small changes, I think the first paragraph needs some more detailed work. The major problem is that the hypothesis is extremely broad and doesn’t reflect what your experiments are actually going to address. [Using Method A] isn’t going to determine whether [very broad hypothesis, worthy of at least one Nobel if eventually proven] is actually true, so you will need to tighten up the hypothesis. Instead I would frame it in terms of your sentence on [description of what you are actually doing] – so something like this:

Hypothesis: that [Outcome B] can be derived by [using Method A], and that this [outcome]reflects the heterogeneity of [Behaviour C] in response to [Condition D].

Feel free to rewrite that sentence! But this is a better reflection of what your experiments will actually be testing.

The hypothesis is currently hidden in the middle of the first introductory paragraph. You need to make it stand out more: put it in bold type, at the end of that first paragraph. I also think it would be better to change the emphasis and order of the sentences leading up to the hypothesis – this very short introduction needs to be intensely focused on the content of the hypothesis and research proposal, with each concept leading logically to the next. At the moment it is a bit choppy, with no obvious connection between adjacent sentences. You need to lead your reader through this section since they might not have the background to make these leaps without guidance.

This order might work, but again please do play around with it until you’re completely satisfied!

1) [Very basic, established fact] (introduce the idea of [Outcome B])
2) This leads to heterogeneity in [Behaviour C]
3) Possible connection between the [heterogeneity of Behaviour C] and response to [Condition D]
4) The problems caused by using [current] approaches that [suck are outdated]
5) Therefore the superiority of [Method A].

Then the hypothesis.

I’d be happy to take another look at this proposal, and especially the first paragraph, once you’ve had the chance to make these edits.

Good luck!

Cath
---------------------

In contrast, my work on the actual proposal sections of the last three reworked / resubmitted grants we put in, with a combined budget well into 7 figures, mostly involved the following:

- Correcting typos
- Standardising the use of alternative spellings (not yet automated, alas)
- Correcting verb/subject disagreements (including "the data is", a pet peeve)
- Correcting preposition use by ESL writers (things like "we will respond from situation X by...")

As I mentioned before, this last round of submissions was not typical in that there were no new grants. But even when I do get involved with new grant applications, the same Catch 22 situation arises:

• I am at my most useful when helping the people who have the least experience in preparing grant applications, and less useful when helping the experienced senior PIs.
  
• The Big Decision (whether to try and find the money to keep me when my contract runs out) will be made by several of the most senior PIs.

What to do?

My strategy is in part adapted from my time in industry, where no job is ever truly safe, and has since evolved to better suit my current job. It is all very different from being a postdoc, when productivity is much more obvious, there are no scary formal reviews, and you know you are SOL after three years anyway!

1) Record keeping.

As I've mentioned before, I have retained the "lab book" habit from my time in the lab. I write all my activities, grouped by project, into a notebook at the end of each day. It's quick and relatively painless; unfortunately the quality of my record keeping tends to suffer in the frenzy of deadline week.

Every month or so I transfer all of my scribbles into an Excel spreadsheet, which contains all of my grants, manuscripts and other projects (for there are many). I have columns for dates, agency/journal, PI, title, funding/acceptance status, and - most importantly - my contribution. This is currently just a string of activities in an unformatted list; I once tried to develop an easy check list system, but with every grant and every PI being different I found it impossible to define consistent categories. Regardless, I should be able to pull out these data (please note correct verb/subject agreement) and insert them into a written progress report or PowerPoint presentation within an hour or two, although I have not actually had to do this yet.

I also archive all received and sent emails, sorted by project, and file hard copies of, for example, the PI's original draft of the grant proposal with my red pen corrections on it.

2) Blowing my own trumpet.

This is the tricky part for me! I mean, I'm English. I would rather keep my trumpet blowing to formal reviews (and blog posts, heh), but I do make myself drop my most significant achievements into conversation (e.g. "Oh by the way, Dr X got her grant" to my immediate supervisor at the end of an unrelated conversation). I hate to contribute to the flood of emails we all get, but I will CC people if appropriate; I copied the student's supervisor (a senior PI who was out of town in the week leading up to the deadline) when I sent the email above, for example.

I've also commandeered the large whiteboard by my desk, which is seen by everyone who visits me and/or my immediate supervisor. As well as a list of everyone's vacation dates and upcoming grant deadlines (its original purpose), it now has a list of "Grants Under Review", with agency and PI listed. Rather than erasing them once the decision is made, I keep them up for as long as I can, complete with an indication of whether they were funded. I just had to erase the older grants at the top of the list to make way for the latest batch of submissions, but there were a good number of successful applications listed up there for the last few months.

Sometimes someone will help me out, for example by copying one of the senior PIs on an email in which they thank me for helping them with their project. This is a rarity though and I can't rely on it.

3) Covering my ass.

This is where my industry experience comes in!

I email a copy of my Excel spreadsheet (see #1) to my Gmail address after every major update. You know, just in case I am terminated without notice and can't access my work files...

I also keep an email folder in my work account labeled "Feedback". Any time I get any significant positive OR negative feedback, it goes in here, and is forwarded to my Gmail account every month or so. I can use the positive feedback to make a case for keeping / reinstating me, and the negative feedback to help me avoid making the same mistake twice. I know I wasn't the only one at my former company who did this!

-------------

I recently described this system to someone with a similar job to mine, and she thought that most of it was a good idea - although I didn't tell her about some of my more paranoid ass covering! Of course she then told me that her reviews to date have been incredibly informal... but then she's on a permanent contract.

Hopefully my system will help me to convince the senior PIs that I'm worth keeping... I'll keep you posted!

Project management in the non-profit research services sector

So what do you do if you think that experience in conducting formal interviews might be good for your future career development, and you also happen to write for an alternative science careers blog?

You interview people with cool alternative careers, that’s what you do! (And you start by practicing on your friends). I get good experience, you get to read about a variety of potential careers, and the interview subject gets a glass or two of wine and half an appetizer. Win-win-win.

For the first article in this series I interviewed “Mermaid”, a good friend, former colleague, and regular commenter on my personal blog, who wishes to remain pseudonymous. Mermaid works as a Project Manager for a grant-funded, non-profit service provider.
___________________

Biological research, in both the academic and industrial sectors, is becoming ever more dependent on specialist high-throughput techniques that are beyond the equipment budget and expertise of the average research lab or start-up company. There has therefore been a trend towards outsourcing these techniques to designated centres. So what's it like on the other side of that fence?

“My analogy is that it’s like ordering a pizza”, says Mermaid.

“The usual pattern is that potential collaborators contact one of our PIs, or come through the website, and they are looking to sequence whatever it is that they have. If I’m lucky they’ll say something like ‘I have this much RNA from this species’, and if I’m unlucky they’ll say ‘how much does sequencing cost?’. So it’s as if someone phones up and says ‘I want three medium pizzas with pepperoni, extra cheese and tomato’, then the next person says ‘I want to feed my friends, how much will that cost me?’”.

Mermaid’s job is to determine exactly how much the pizza will cost, and how long it will take to be delivered, and to go back and forth with the potential collaborator until both parties are happy.

The first step is to communicate with each researcher to determine exactly what they want to do. The research outsourced to the sequencing centre covers a wide variety of fields; from cancer research to wine grape genomics, from anti-mountain pine beetle programmes to crop research. This variety obviously appeals strongly to Mermaid, whose academic and industry background includes such diverse fields as marine biology, medical genetics, biochemistry and stem cell biology. “I like that it’s all different, and what I really like is talking to people, finding out what they do, talking to them about their research”. The consummate scientist, she ends the long list of projects she’s been involved in with the words “it’s all very cool”. But has she had the chance to go back to her marine biology roots, which she originally abandoned after taking on co-op projects that involved “picking up dead fish on salmon farms”? With a smile she says “I think there are some salmon projects, but I haven’t had to deal with them!”

Once Mermaid has a good grasp of what the researcher wants, she helps them to choose the most appropriate sequencing service. “The effort depends on how much they understand about the different technologies”, she explains. “If it’s someone we’ve worked with before it’ll be one email, if it’s someone who’s got no idea it could be twenty”. She then factors in variables such as the type and scope of the collaboration, the number and size of the collaborator’s samples, and the technology used, to generate a rough cost quotation. If the client is still interested, she delivers a formal quote and estimated turn-around time. There are often several rounds of back and forth until the collaborator is happy, at which point Mermaid passes the work order on to the centre’s other departments.

When she’s not triaging pizza orders, Mermaid is also starting to manage specific projects, such as a grant held by one of the centre’s PIs. “I’ll be the first point of contact for questions about budgets, timelines, things like that”, she says. “I’m the one that makes the information flow”. This role includes coordinating monthly meetings, and tracking the project’s budget. “So I might have to say ‘you’ve only got enough money for 15 samples, if you want to do 20 we’ll have to make cuts elsewhere’, things like that”.

Mermaid moved into her current job fairly recently, following six years working in industry. Her previous job was the same as mine, with overlapping but distinct “good, bad and ugly” lists. Like me she saw no clear way to progress any further within the company, and realized that it was time to look for something else.

Using her network to good effect, Mermaid set up an informal chat with a contact in the non-profit research sector. She went into the meeting with no specific goals. “I was exploring how my skills might match anything in biotech, I wasn’t specifically looking in that sector”, she remembers. “And I knew this person had contacts in a wide variety of places”. However, she left the meeting with the name and contact details of a senior project manager who was hiring at the time. “I asked to meet her for coffee, and again I was just information gathering, looking for the kind of training that might be needed, finding out more about the field”. Soon after this informal discussion, Mermaid was encouraged to apply for her current position.

The multiple formal job interviews, with different people and departments, focused not only on Mermaid’s background knowledge and skill set, but also on how she approaches problems. “They would ask things like ‘if this situation would arise, how would you deal with it?’”, she says. “Knowing full well that you would have no idea what the official approach would be. But how do you think, how do your thought processes work?”.

The interviewers clearly liked what they saw in Mermaid, who has already been promoted after only a few months in the job. But what skills, other than a varied research background, would someone have to demonstrate if they wanted to follow in her footsteps?

“You have to be organized, you have to be on top of the paperwork, it’s just crazy”, she says. “People are managing three or four PIs, and you also have to be a little bit sensitive to them”. “So you shouldn’t go into this if you’re a bitter grad student who hates all PIs?” I ask. “You shouldn’t go into this if you can’t just laugh”, says Mermaid with an appropriate twinkle in her eye.

There is one final important point for anyone wishing to move into project management. Mermaid tells me that since she started her job, several people have interviewed unsuccessfully for similar positions, in part because they said that they wanted to leave research. “A project manager does not leave research”, says Mermaid. “Your job is to ensure that the research stays within the scope of the grant and its budget. You’re not leaving research, you’re an integral part of it”.

______________________

I know Mermaid reads this blog from time to time, and will probably be very interested in the feedback on this post! So if you have any questions for her, please leave them in the comments.

How I landed my alternative career in science writing

When I first started talking to people and reading about alternative science careers, I quickly realized that there is no set path. People find themselves in editing, writing, or policy positions by any number of round-about ways. Personal contacts are often key. This can make the prospect of “breaking into” an alternative career quite daunting. Academia is hard, but at least we’re all familiar with the setting, and we have at least some idea of what one needs to do to “succeed” i.e. land that tenure-track job. If you’re a grad student or postdoc, you have a built-in network of academic contacts and support. And you have probably seen, or are seeing right now, colleagues applying to and interviewing for academic positions.

But what if your dream is to go into science policy? To be a science journalist for the New York Times? To be a biotech consultant for investment firms, or to produce scientific documentaries for the Discovery channel? How the hell do you get into that?!

No one follows the exact same path. There isn’t a degree program you can take that will automatically get you the job you want. My own journey to an alternative career has been marked by meanderings and false starts. But I thought that it might be helpful in this forum, to some reader somewhere, to give a detailed account of how I’ve landed where I now am.

*****************************

Like probably 90% of my grad school cohort, I thought I wanted to head up an academic lab and spend my life doing basic research. I got Ph.D. in the biomedical sciences and then landed a good postdoc. Unfortunately, that Nature paper never appeared, and my postdoc advisor lost a major source of funding midway through my time in his lab. When I returned to the lab after a maternity leave, I was informed that my contract would be up in one year. Until this time, I had lived in a bubble, humming along at the bench with no real thought for my future other than a vague hope that the next breakthrough experiment was just around the corner. I had given no real thought to my career plan, and certainly had not researched any type of alternative career. But nothing sharpens the mind like the imminent loss of a job.

Option #1: Go into teaching

I knew that I wasn’t competitive for tenure-track research positions, and I didn’t particularly care for doing a second postdoc. So I decided that I wanted to be a teacher. That fall, I applied for tenure-track teaching positions. Saddled with the classic “two-body problem,” I restricted my search to schools that were in commuting distance. This turned out to be a grand total of two. I had a preliminary phone interview with one of the institutions, and it was so immediately clear that the position was unsuited for me that I was quite relieved to let that particular job go. The other job, at a regional state university with masters degree programs but no doctorate programs, seemed far more promising, and I was invited for a campus interview. At this point, I had practically ZERO college teaching experience (aside from eight weeks of TAing in grad school). I knew nothing about the environment at a teaching-oriented school or regional state university. And some snotty, elitist part of me still saw this path as second best.

I interviewed for the job and SURPRISE! I loved the school. I adored the people and environment. I loved it that people seemed excited about my research, and that I would have lab space and funding to continue my interests (even if only at a level that would be laughable to those at an R1). I decided that this was my dream job.

I didn’t get the dream job.

Option #2: Frantically apply for any other science-related jobs in the area.

I found a grand total of two. Postdoc School was not exactly in a biotech hub, and industry positions were extremely limited. I applied for one position in technology transfer and one position in medical writing. Didn’t get interviews for either.

Option #3: Drop out and get knocked up.

Um, this is the one I went with.

Seriously, my husband and I wanted a second baby. And it seemed that this was a good time to “opt out” and re-evaluate what I really wanted out of a career and life.

While working on the “knocked-up” part and caring for my toddler daughter, I kept exploring career options. Realizing that I needed more teaching experience if I actually wanted to be competitive for teaching positions, I signed on to teach a course at Regional State University, the same school that I had interviewed with and loved. And a friend at a medical communications company promised to send freelance science writing/editing jobs my way.

I learned that Regional State University was indeed a great place to work and teach. I also learned that I didn’t particularly care for teaching. I found the pressure of preparing new lectures each week, week after week, trying to master and present myself as expert in a new subfield each time . . . tiring and stressful. I liked getting to know students, and I liked working with them one-on-one. But I couldn’t see myself teaching full-time for a living, for years on end.

On the other hand, I did enjoy the medical writing/editing that I did for my friend’s company. My friend gave me a contract assignment reviewing and editing the text for a course in oncology. It was lucrative (far more than teaching!) and it was fun.

Trailing spouse

Unbeknownst to me, my husband, an M.D/Ph.D, was going through his own career crisis at the time. Out of the blue (it seemed) he announced to me that he was looking for a full-time clinical position. But, but, I sputtered in disbelief. You just set up your own lab! You just got your own lab going this summer! But my husband was disillusioned by academic research, and had been secretly dreaming of an escape hatch.

Luckily, a clinical position in his field of medical expertise opened up at a hospital across the state, in a city very near my parents’ home. Husband interviewed for the job and got it. Do you mind moving? he asked me. Hell, no. We’d be close to my parents, and there was nothing keeping us in Postdoc City. I was heavily pregnant with our second child by this point. We moved to the new city, and I gave birth just one month later.

I became a stay-at-home mother for the next year. During this time, I continued to do occasional freelance medical writing/editing projects for my friend’s medical communications company. I also started to seriously investigate medical writing and editing as a full time career. I read everything I could find on the Web (This article and this at Sciencecareers.org are good places to start). Eager for more work, I haunted the job board at the Council of Science Editors and applied for contract jobs I saw advertised there. Now it happens that there are numerous scientific manuscript editing services out there on the Web (just Google “scientific editing” to see what I mean). One such editing service company responded with interest to my application and asked for some editing samples. Thanks to my work with my friend’s company, I actually had samples on hand. The samples helped land me the job, so I was now a contractor with a second company. Through my friend’s company, I gained experience writing for the pharmaceutical industry. Through the second company, I gained a little more experience editing academic scientific manuscripts and grant proposals.

I had not quite, however, given up that old dream of a life at the laboratory bench. In fact, at the back of my mind, I harbored a fantasy of a triumphant return to the bench. I knew that the NIH runs a program of grant supplements to promote the reentry of women into biomedical research who have taken time out of their research careers for the sake of family responsibilities (These grant supplements are applicable to men as well, despite the name. Also, the sponsoring lab needs to be funded by an RO1 or one of a few other select NIH grants). Our new city of residence had no biotech industry, but it did have an Up-and-Coming Research Institute associated with my husband’s hospital. I regularly browsed this institute’s web site for information on its research labs, and for news and job opportunities. From time to time interesting postdoc positions popped up, but I wasn’t quite ready to apply. And then one day, when my youngest had already turned one year old, a new job posting popped up on Institute’s web page. It was a job posting for a Scientific Writer. And it sounded perfect.

I applied for the job. And I also applied for a postdoc. I still wanted to keep my options open, and why not, as this was just the application stage? I still thought that maybe I did want to head back to the bench, and the research position was in a lab whose research interests were loosely related to the avenues I’d been exploring during my first postdoc.

I received initial interview offers for both the science writing job and the postdoc. Then the postdoc position was eliminated when the PI realized she didn’t have the funds. The interview process for the writing position was delayed as that PI struggled for funds. I went through an angsty period and wondered if I should apply for other postdocs at the institute. But the other advertised postdoc positions were not of real research interest to me, and I didn’t have the stomach to apply for them. I waited. Eventually, I did get an interview for the science writing job. And so here I am today.

I am now a scientific writer and editor for a large laboratory at a private, nonprofit biomedical research institute. My position is unusual in that I do not work for an entire department (like Cath) or institute, but rather for one individual laboratory. My major responsibilities are to edit and help prepare grant proposals and scientific manuscripts for publication. So far, I am truly enjoying the work. I remain close to science—I interact daily with scientists, work with exciting, unpublished data, attend lab meetings and seminars, and read, read, read. To my surprise, I’ve found that I have yet (so far) to pine for the laboratory bench. Although earlier this week I was so bored (we’d just passed on a potential grant proposal, and things were slow) that I did offer to do DNA minipreps for anyone who wanted!

The key to landing my current position, I think, was the freelance scientific writing and editing work that I had done. Grad students and postdocs get writing experience as part of their standard training, but I had something extra in my resume, something that distinguished me and that suggested a special commitment to writing and editing. You can bet that I played that up in my interview. You can bet that I played up my English minor in college, as well =)

Now, having landed where I am, will I stay here? I don’t know. I might not be done with my meandering ways. But you can bet that, as Cath has suggested before, I will keep looking for marketable experience and my next potential job. Especially in these economic times, I think we all need to stay flexible and always be prepared to look for that next job. In my case, this may be as early as next year. The funding for my position is only guaranteed for one year. But in the meantime, I am making contacts at my research institute and am getting valuable new experience in grantsmanship. I am feeling more hopeful about my job prospects than at any time in the past year, when I really thought my science career was doomed.*

*I guess my “traditional” career as a tenure-track researcher is doomed, but a career in science still is not.*