--Updated Post--June 15, 2011

I wanted to repost my comments from June 2010 at a National Academies' conference on how to increase innovation in federal statistics as the final report from the National Academies of that event is now available.  They picked up on some of my comments on user innovation, as did some of the agencies in the room.  I am pleased to report that some of the concepts I raised at the event have been implemented in different agencies - most concretely the idea of putting notices of opportunities to comment closer to where users are getting their data (Business R&D and Innovation Survey from NSF and Census is latest).  These are some small but good changes.  I still worry that the concreteness of that recommendation came through but some of the spirit of it might not have.  I'm going to be mulling over some of these concepts for an upcoming presentation I am contributing to, coincidentally at the National Academies, so will be thinking more on the subject again.  

As for the final report, I think there are some good things in it, but I must admit I was very disappointed in the event.  It was a discussion of innovation led almost entirely by the people who are already heavyweights in the system and that made range of the discussion very narrow.  I didn't see much of an outsider perspective.  The statistical system's real concern with innovation should be that they have so few interested parties in the room.  At that event there were a few of us not from the system as speakers but that was about it.  The room was packed with agency representatives.  Every major industry association and business group should have been concerned with that meeting if it was actually relevant because it'd mean that the system was trying to find ways to be relevant.  As I found the meeting, it was a discussion of how we can be better at things we probably should have been better at 10 years ago; not how we can be an innovative statistical system that measures the pulse of the U.S. and global economies and sets the tone for what innovation means in statistical systems around the world.  

--Original Post--June 28, 2010

Tuesday I will be a discussant at the National Academies’ Committee on National Statistics workshop in Washington, DC, looking at how to facilitate innovation in federal statistics.  It’s a topic I am passionate about and unfortunately it’s an area of interest where we often feel like soloists rather than members of a choir.  As we raise our calls for improvement and the need to better measure business activity we need more charitable foundations, academics, businesses, and local economic development officials at the table.  If it weren’t for the tireless work of groups like the Association for Public Data Users (APDU), I fear that the meager levels of innovation which currently exist in the system would be next to nonexistent. 

Desire for Internal Innovation.  In our experience dealing with entrepreneurship and innovation measurement, I’ve seen a lot of personnel within specific agencies that want to see more innovation in their products.  Sadly, too often these personnel have no forums or opportunities to explore possible improvements and few incentives that would encourage innovation.  More than any funds we’ve spent, our efforts to spur improvements in federal data on entrepreneurship and innovation have come mostly through convening of symposiums and workshops that allow agencies to present their plans for improvement, debate with academic and agency experts, and see what other exciting developments are occurring in academic and agency environments.  The returns to this dialog seem only to increase over time and seem well-placed for topics-based statistical discussions.    
   

Users Can Drive Innovation.  When you provide basic information, like the statistical agencies often do, one of the saddest parts of the process is that much of the information goes on to uses which you never become aware.  As a major producer of data, at Kauffman, I know how excited I get to see some of our data used in an analysis or something new but too often the public doesn’t circle back with this material.  We’ve seen in our own experience that if you create events, online platforms, and other vehicles to show your support of users that it can result in a positive feedback loop and improved data quality.  Through the Kauffman Firm Survey, we’ve undertaken an annual open call for suggested improvements in the data we collect which has over the years led to major improvements in the usability of the data for research and analysis.  Had we not undertaken this process of trying to interact with the users of our data we would have missed significant improvements in data collected on the finances of new firms which are going to greatly improve our analysis of the current recession. 

Sadly, most agencies know few of the users of their data, and even where they know them they are often hesitant to interact and hear their needs, feedback, etc.


Interaction, Interaction, Interaction.  I am not sure how many ways I can say it but I’ll say it one more – interaction.  The thing which I find most disappointing with the current statistical system is that individuals within agencies often want more interaction with each other and other experts on their topical collections but for the large part this does not happen.  Money to travel to conferences is very limited and the interagency groups and agencies overseeing some of this process only have opportunities for interaction at very high levels, not on more granular areas of interest and need.  At Kauffman we try to bring academics into the conversation but know that from trying to do this that it’s very tough.  Key constituencies like state representatives and sub-state representatives don’t have formal bodies where the agencies can receive feedback on possible data publications and so they don’t hear what these groups actually want.


Push Ourselves First.  As someone who takes time to read most notices about data collection on the Federal Register, I applaud the few efforts that actually submit meaningful requests for feedback and are open to feedback submitted.  In reality these processes are largely a formality that is hidden away so that as few people as possible find them.  Even submitted comments following outlined procedures can be dismissed as “it’s too late in the process.”  Let’s stop this self-perpetuating process of mediocrity.  If we are going to seek comment, seek comment. 


I’ve offered a lot of criticism here but I recognize that we at Kauffman still have a lot to learn in this regard.  One of the things are beta testing is this concept of a Data Wiki for the Kauffman Firm Survey.  It’s not something which was easy for us to do and opens us up to user feedback, positive and negative, in a much greater way that we’ve done before.  But this is also an area where we believe what we learn can help to inform the statistical agencies. 

In my opinion, until OMB and the agencies become more genuine in their efforts to draw data users into the conversation we have little room to complain about lack of funding.  These are things within their power to change.  We cannot afford to idly hope for improvements in national statistics.  I hope others out there agree with me and will add your voice to the discussion. 

The two surveys have different definitions and different methods, but general trends can be seen in both surveys.

• Of all the industries that contain green jobs, the construction industry has added the most green jobs out of any other. These construction jobs are those that are either geared to energy efficiency or are related to renewable energy.
• Green jobs have a higher than average wage, and many green jobs don’t require a bachelor’s degree. However, the highest paying green jobs do require higher education.
• Individual occupations are becoming green, but the often-talked-about green industry is generally non-existent. Most green jobs today are old jobs with a green focus. Few are brand-new, and just about all of them retain the same title as before (exceptions are those like wind turbine technician). Both researchers refer to this as the “greening” of the economy.

There was a lot of advice given on how to run a survey and how the researchers might have run their own surveys differently. Major points break down as follows:

• The Bureau of Labor statistics has not issued a standard definition of a “green job” yet, but there are already a lot of guidelines out there (both from surveys like these and other sources). If a researcher were to develop his/her own definition, a good jumping off point would be to examine a paper put together by the Workforce Information Council which can be found here.
• As stated above, there is no specific green area of the economy; all parts of the economy are experiencing “greening” to some extent. Therefore, in future studies, individual jobs must be the unit of analysis. Aggregating to the firm level (or even across occupations with the same title) will tend to bias results.
• Response rates of surveys mailed to firms (who then tally green occupations) are typically low. Both surveys had response rates of about half. Therefore, when conducting future surveys, substantial follow-up efforts are necessary to try to minimize non-response bias.
• Other ways to increase the response rate are to: