Research Impact in July

July’s Research Bites programme is all about Research Impact. Just turn up, no need to book!

REF Impact 101

Monday 4th July, 12pm, Bowland North SR 19
An overview of what is considered ‘research impact’ according to the new Research Excellence Framework (REF), and what is eligible for submission.
Amy Gibbons, Faculty Impact Manager, Faculty of Science & Technology

Collecting and curating impact evidence

Thursday 7th July, 12pm, Bowland North SR 19
Why it’s important to gather evidence of research impact, identifying what constitutes evidence, and how best to collate and store it.
Jacqueline Young, Impact Officer, Research Services

Writing ‘Pathways to Impact’ statements

Wednesday 13th July, 12pm, Bowland North SR 19
What they are, why they’re important, and some tips on best practice.
Ross Dachraoui, Impact Officer, Research Services

Proactive partnerships for impact

Tuesday 19th July, 12pm, Bowland North SR 19
Co-designing productive impact driven research.
Nick King & Colin McLaughlin, Business Partnerships Team, Faculty of Science and Technology

Recording impact using Pure

Thursday 21st July, 12pm, Bowland North SR 19
Use the University Research Information System, Pure, to record your research impact.
Claire Tinker-Mill, Research System Administrator, Research Services

Measuring academic impact using citations and bibliometrics

Tuesday 26th July, 12pm, Bowland North SR 4
Learn how citations and bibliometrics can be used to indicate academic impact.
Tanya Williamson, Academic Liaison Librarian, Library

What SciVal can tell us about academic impact

Thursday 28th July, 12pm, Bowland North SR 4
Learn how this powerful research analytics tool can give you insights into academic impact.
Masud Khokhar, Head of Digital Innovation, Library

Cultures of counting: Metrics through a critical lens

Some Research Bites attendees may be interested in the Cultures of counting: Metrics through a critical lens seminar on Tuesday 24th May, 1-3pm.

In this seminar, James Wilsdon, who chaired The Metric Tide, will outline its main findings, and reflect on ongoing efforts to influence debates about UK research policy and funding, including over the design of the next Research Excellence Framework (REF), which is currently the focus of a further review by Lord Stern.

Paul Ashwin, in the context of the proposed TEF, will examine the challenges of developing measures of teaching quality that do not simply reflect institutional prestige.

Both speakers will consider what a culture of ‘responsible metrics’ might look like for research and teaching, and the opportunities and obstacles to achieving this.

Read more…

This talk is part of the Designing the Academic Self series, sponsored by the Academics Writing project and the Northwest Doctoral Training Center and is open to early career researchers and doctoral students from Lancaster, Manchester and Liverpool universities.

To book your place, visit Eventbrite. 

Bang those Robots’ Heads together

Thanks to those of you who attended the Introduction to Citation Analysis at the start of July. The session was longer than advertised, and I appreciated you staying until the end!

The session covered:

  • What is citation analysis
  • What is it for?
  • How do I analyse citations? – Web of Science and Scopus
  • What’s wrong with citation analysis?

The slides are available here:

and if you wish, you can watch the full recording of the session. NB. Recordings work best using any browser other than Google Chrome.

Research Bites in July

wetlands-mother-of-pearl-butterfly-342160_1280

This month’s theme is Research metrics.

Join us for an informal 20 minute session. Just turn up, no need to book.

‘Bang those robot heads together’
Thursday 9th July, 12.00. Bowland North SR 19
A brief introduction to the citation analysis tools available in Web of Science and Scopus.
Tanya Williamson, Assistant Librarian, Library

The Journal Impact Factor
Monday 13th July, 12.00. Bowland North SR 19
Understand the importance of Journal Impact Factors for you, the University and your funding body. Discover the most common metrics, and address your concerns.
Jenny Brine, Subject Librarian, Library

More than your h-index?
Thursday 16th July, 12.00. Bowland North SR 19
Learn about the range of tools/metrics that you can use to help understand the impact of your work.
Tanya Williamson, Assistant Librarian, Library

Altmetrics: beyond the impact factor
Tuesday 21st July, 12.00. Bowland North SR 19
A look at altmetrics and article-level metrics, and their growing significance to researchers.
Hardy Schwamm, Research Data and Repository Manager, Library

The power of SciVal
Thursday 23rd July, 12.00. Bowland North SR 19
An overview of this powerful research evaluation tool will be of interest to researchers and research managers.
Masud Khokhar, Head of Digital Innovation, Library

Beyond the Impact Factor: Altmetrics

Many thanks to Hardy Schwamm for today’s Research Bite introducing altmetrics, and also to those of you who came along and contributed your questions and thoughts to the discussion.

Context

In the context of huge increases in publication output, the Altmetrics manifesto claims that we need new ways to filter for quality. The development of altmetrics is also an attempt to catch up with the changes in scholarly communication. Altmetrics are a reaction to the fact that the impact factor is often incorrectly used to assess the impact of individual articles, when in fact it just evaluates the impact of the journal. Altmetrics also acknowledge that research outputs are broader than just journal articles, and encompass datasets, software, blog posts, presentations and so on.

Altmetrics are:

Metrics based on the social web

Among the most promising and influential Altmetrics services are:

  • Altmetric – UK based, not affiliated to a publisher, focusing on article-centric or article level metrics
  • PlumX – gather metrics on different ‘artifacts’, but also on individuals, groups and institutions
  • ImpactStory  – focusing on individuals

The key audiences are:

  • Individuals can use altmetrics to understand their own impact, or influence.
  • Publishers can use it to inform readers, marketing and identify strengths
  • Institutions can use it to gauge impact, inform REF and funding bids and recruitment?

Not everyone is happy with incorporating the Almetric score. Can you really compare across disciplines? Is an article with a score of 2000 really that much ‘better’ than one scoring 25? Altmetric does include percentile information when you click through to the detail. This ability to click through to the detail, and actually delve into those ‘mentions’ is a useful feature.

Questions

How do they track your output (e.g. article)? Altmetric use the DOI (Digital Object Identifier), PubMed ID and ArXiv ID to track mentions.

Does it count mentions of your name? Altmetric just looks at articles, not the identity of the researcher. ImpactStory and PlumX do focus more on the individual, but it would be in association with a particular work, as far as we understand. So an appearance on TV wouldn’t be counted (unless the broadcaster had a link on their website too).

Does it only measure positive mentions? No. There’s no discrimination, so like traditional citation counts, others could be denouncing your research, but you’d still benefit from the stats!

Is there going to be an institutional licence to the product? The Library and RSO are aware of altmetrics and inviting providers to visit and tell us more.

Hardy: Would this kind of product be useful? Is the score useful? Participants: It’s all ‘grist to the mill’. If you can provide your line manager with some data like this it could contribute to them understanding your performance. The score seems pretty meaningless though. It’s good to have a service which aggregates all this data in such a timely way. Citations can take a long time to accrue. It bridges the gap between informal discussions and citations in journal articles. Though it can never be a complete measure. Altmetrics could inform the REF in terms of impact of research in the community or public engagement outside of the academy.

Is it just a snapshot in time (i.e. does it only provide data from Twitter for 30 days?) It appears that it is not just a snapshot in time, but a record of mentions, though data collection would presumably have to start somewhere, so may not be accurate for older papers (by older, I mean pre-2011!).

Do altmetrics cover patents? Plum covers citation in US patents.

What’s our definition of impact? Is it just getting your research out there and generating a buzz? Some disciplines view this kind of attention as public engagement. Impact is making a change to the world, e.g. through policy change or a technological advance. Altmetrics contribute to a multi-faceted view of impact, including non-academic engagement.

Can you track the geographic element? Yes, with Altmetric. When you click through to the detailed data. e.g.

altmetriceg

Is it performing as a filter? Ranking efforts and scores like in Altmetric are difficult to interpret and shouldn’t be compared across disciplines. However, if you see that an article was discussed in the (social) media it might be worth seeing why it got mentioned so often.

Won’t it be open to scholars artificially inflating scores? Yes, just like traditional citation practices! It also favours researchers who are active on the social web.

The Altmetrics manifesto disagrees:
Some have suggested altmetrics would be too easy to game; we argue the opposite. The JIF is appallingly open to manipulation; mature altmetrics systems could be more robust, leveraging the diversity of of altmetrics and statistical power of big data to algorithmically detect and correct for fraudulent activity.

This post was amended on 24/09/2014 following feedback from Hardy.

Research Bites sessions in September 2014

Join us for an informal 20 minute session. Bring your lunch, and a friend.

Just turn up, no need to book.

Filing a patent on your invention and commercialising it
Thursday 4th September, 12.00. Fylde C48
Expert advice on filing a patent from the University’s Research Enterprise Service.
Gavin Smith, Intellectual Property Officer.

Non-patent Intellectual Property
Friday 12th September, 12.00. Fylde C48
Expert advice on intellectual property from the University’s Research Enterprise Service.
Gavin Smith, Intellectual Property Officer.

Beyond the Impact Factor: Altmetrics
Friday 19th September, 12.00. Fylde C48
A look at altmetrics and their growing significance to researchers
Hardy Schwamm, Research Data and Repository Manager

Your identity as a researcher – ORCID
Friday 26th September, 12.00. Fylde C48
Find out what ORCID is, and how it can help you to distinguish your work from someone else’s.
Tanya Williamson, Assistant Librarian.