Reinventing public R&D: patent policy and the commercialization of national...
Despite their magnitude and potential impact, federal R&D expenditures outside of research universities have attracted little economic scrutiny. We examine the initiatives since 1980 to encourage patenting and technology transfer at the national laboratories. Both field and empirical research
1. Introduction
* The United States government is by far the single largest performer and funder of research and development in the world. Between 1941 and 2000, the U.S. government spent $2.7 trillion (in 2000 dollars) on R&D, just under one-half of the total amount undertaken in the United States (see Figure 1). In 1993, the government's R&D expenditures represented about 18% of the total funding of R&D in the major industrialized countries (National Science Board, 1998).
[Figure 1 ILLUSTRATION OMITTED]
These expenditures, despite their magnitude and their potentially profound impact on productivity and growth, have attracted surprisingly little scrutiny by economists. Although one environment in which federally funded research takes place, the research university, has been studied extensively,(1) it accounts for a relatively small percentage of overall funding. (Between 1955 and 1997, only 24% of the total federally funded R&D performed in noncorporate settings took place in academic institutions.) The majority of these activities took place in laboratories owned by such agencies as the Departments of Defense and Energy, the National Aeronautics and Space Administration, and the National Institutes of Health. R&D activities in this arena have attracted little academic scrutiny.(2)
Lending particular urgency to such research is the interest shown by policy makers, both in the United States and abroad, in increasing the role of government-owned laboratories in the technology commercialization process. Beginning in 1980, a series of legislative initiatives and executive orders in the United States have sought to encourage the patenting and the licensing of federally owned technologies, as well as the formation of cooperative arrangements between laboratories and private firms. These challenges have been particularly pressing at the laboratories devoted to national defense, whose primary historical mission of designing and testing nuclear weapons has been rendered largely obsolete by world events. Similar efforts have been launched in many other nations.
We examine whether the statutory changes of the 1980s have had a significant impact on technology transfer from the national laboratories. We study the subset of federally funded research and development centers (FFRDCs) owned by the U.S. Department of Energy (DOE). These include some of the largest R&D laboratories in the country, such as the Lawrence Livermore, Los Alamos, and Sandia facilities. We undertake case studies of two particular facilities, and then we employ a series of databases developed by DOE not hitherto examined by academics. Both analyses explore how patenting, the utilization of these patents by industry, and other technology-transfer activities have shifted in response to these legislative changes.
In addition to examining shifts in the overall level of technology-transfer activities, we explore how the heterogeneous features of these facilities affect their success in commercialization. These laboratories differ from each other in at least three critical respects. First, the quality of the laboratories' technology may differ. This may partially reflect the nature of the laboratories' missions: those specializing in basic science or defense-related technologies, for instance, may have fewer technologies ripe for commercialization. It may also reflect the breadth of the laboratories' activities. Second, the facilities differ in the nature of the political problems that commercialization efforts are likely to face. The conditions under which contractors should be permitted to patent and license federally funded technologies remains highly controversial.(3) Across the laboratories, the relationships between the contractors assigned to run the facilities and the Department of Energy differ significantly. Finally, the locations of the facilities are highly disparate. Some FFRDCs are located near population centers with extensive innovative and entrepreneurial activities, while others are in highly remote areas. An extensive literature has documented the regional aspects of knowledge spillovers, which might suggest that laboratories differ a great deal in their ability to have commercial impacts.
The results challenge some of the conventional wisdom about these laboratories, as reflected in government studies and press accounts. Both the case studies and empirical analyses suggest that the policy changes of the 1980s, far from having no effect, appear to have had a substantial impact on the patenting activity by the national laboratories. At the beginning of this period the laboratories had considerably fewer patents per R&D dollar than the average university, but today they are about equal. Even more impressive is the evidence from the citations in other patents, a proxy for their importance. While the recent increase in university patenting has been accompanied by a substantial decline in the quality of the awards (as measured by the number of citations they receive), the quality of the laboratory patents has remained constant or has increased slightly. These results suggest the organizational structure of the government-owned contractor-operated model used at DOE laboratories may be far more credible than critics have suggested.
The relatively small number of DOE laboratories, combined with limited variation over time for each lab, makes it difficult to draw strong conclusions about how the different laboratory environments and organizational structures affect the technology-transfer process. But several findings can be highlighted. First, consistent with claims that the diversification led to a degradation of the quality of the laboratories' R&D, it appears that the greatest commercial activity has derived from laboratories that have remained focused. Second, facilities with a turnover of contractors, when pressures from parties resistant to exclusive licensing are likely to have been lowest, have had greater success in accelerating their rate of commercialization. The case studies also present evidence consistent with these conclusions.
The plan of this article is as follows. In Section 2 we provide some background information on government patent policy and the national laboratories. Section 3 presents two brief case studies of particular laboratories. Section 4 describes the construction of the dataset and presents the analysis. Section 5 concludes.
2. Patent policy and the national laboratories
* Patent policy and federally funded R&D. A substantial literature discusses federal policies toward the patenting and commercialization of the innovations whose development it has funded (Cohen and Noll, 1996).(4) Even a casual review of these works, however, makes clear how little the debate has changed over the decades. Many advocates have consistently called for government to take title to innovations that it funds, to ensure the greatest diffusion of the breakthroughs. Others have argued for a policy of allowing contractors to assume title to federally funded inventions or, alternatively, allowing the exclusive licensing of these discoveries.
Questions about the federal government's right to patent the results of the research it funded were the subject of litigation and congressional debate as early as the 1880s, but the debate assumed much greater visibility with the onset of World War II. The dramatic expansion of federal R&D effort during the war raised questions about the disposal of the rights to these discoveries. Two reports commissioned by President Franklin D. Roosevelt reached dramatically different conclusions, and they framed the debate that would follow in the succeeding decades.
The National Patent Planning Commission, an ad hoc body established shortly after the Pearl Harbor attack to examine the disposition of the patents developed during the war, opined:
It often happens, particularly in new fields, that what is available for exploitation by everyone is undertaken by no one. There undoubtedly are Government-owned patents which should be made available to the public in commercial form, but which, because they call for a substantial capital investment, private manufacturers have been unwilling to commercialize under a nonexclusive license (U.S. House of Representatives, 1945; p. 5).
A second report, completed in 1947 by the Department of Justice, took a very different tack. Rather, it argued that "innovations financed with public funds should inure to the benefit of the public, and should not become a purely private monopoly under which the public may be charged for, or even denied, the use of technology which it has financed" (U.S. Department of Justice, 1947, Vol. 1, p. 2). The report urged the adoption of a uniform policy forbidding both the granting of patent rights to contractors and exclusive licenses to federal technology in all but extraordinary circumstances. Over the ensuing 30 years, federal patent policy vacillated between these two views (Forman, 1957; Neumeyer and Stedman, 1971).
Beginning in the 1980s, policy seemed to shift decisively in favor of permitting exclusive licenses of publicly funded research to encourage commercialization. The Stevenson-Wydler Technology Innovation Act of 1980 (P.L. 96-480) explicitly made technology transfer a mission of all federal laboratories and created a variety of institutional structures to facilitate this mission. Among other steps, it required that all major federal laboratories establish an Office of Research and Technology Applications to undertake technology transfer activities. At about the same time, the Bayh-Dole Act allowed academic institutions and nonprofit institutions to automatically retain title to patents derived from federally funded R&D. The act also explicitly authorized government-operated laboratories to grant exclusive licenses on government-owned patents.
These two acts were followed by a series of initiatives in the 1982-1989 period that extended and broadened their reach. In 1986, the Federal Technology Transfer Act allowed government-operated facilities to enter into cooperative R&D arrangements (CRADAs) with industry (P.L. 99-502), as well as to grant outside collaborators the title to any invention that resulted. In 1989, the National Competitiveness Technology Transfer Act of 1989 (P.L. 101-189) extended the 1986 legislation enabling the formation of CRADAs to government-owned contractor-operated facilities. See Table 1 for a summary of relevant legislation.
TABLE 1 Key Federal Technology Policy Initiatives Related to
National Laboratory Technology Transfer
Year Title Description
1945 National Patent Proposed that agencies be allowed
Planning Commission to set disparate policies regarding
technology.
1947 U.S. Department of Urged adoption of uniform policy in
Justice Investigation which the government took title to
of Government Policies almost all federally funded
and Practices technologies.
1950 Executive Order 10096 Established centralized Government
Patent Board to decide upon patent
ownership.
1961 Executive Order 10930 Allowed agencies to set separate
patent policies, and to sometimes
grant nonexclusive licenses.
1980 Stevenson-Wydler Made technology transfer a goal of
Technology Innovation Act all federal laboratories; required
each to set up technology-transfer
office.
1980 Patent and Trademark Allowed academic and nonprofit
Laws Amendment institutions to retain title to
(Bayh-Dole) Act federally funded R&D.
1984 Trademark Clarification Extended many, but not all,
Act provisions of Bayh-Dole Act to
contractor-operated laboratories.
1986 Federal Technology Allowed government operated
Transfer Act laboratories to enter into
cooperative R&D agreements (CRADAs)
with industry.
1989 National Competitiveness Allowed contractor-operated
Technology Transfer Act laboratories to enter into CRADAs.
This wave of legislation did not resolve the debate over how much ownership of government-funded R&D ought to be transferred to private-sector entities. Congressional and agency investigations of inappropriate behavior during the commercialization process--particularly violation of fairness of opportunity and conflict of interest regulations during the spin-out and licensing process--continued to be commonplace. CRADAs and other efforts to allow government to work with large companies have remained controversial. Nonetheless, this set of legislation represented a decisive shift in the long debate on government patent policy.
[] The DOE FFRDCs. Many of the DOE FFRDCs,(5) also known as national laboratories, had their origins in the Manhattan Project during World War II. The development of the atomic bomb required the establishment of a number of specialized facilities, many of which were located in remote areas because of concerns about safety and security. After the war, these facilities were placed under the control of outside contractors, a mix of universities and private firms. It was hoped that these government-owned contractor-operated facilities (GOCOs) would be insulated from political pressures and would be better able to attract and retain talented personnel because they did not have to conform to civil service rules. From the launch of these facilities until today, however, the overwhelming majority of the funding for the programs has come from the federal government. In particular, almost all of the funds have come from the primary energy agency (first the Atomic Energy Commission, then the Energy Research and Development Administration, and finally the Department of Energy) and the units of the Department of Defense responsible for nuclear weapons design and procurement.
A series of reports over the past several decades have highlighted the limitations of the GOCO model.(6) They have repeatedly highlighted the same problems with the management of the laboratory system: (1) the desired political independence has never been achieved, with DOE imposing extensive regulatory guidelines that limit contractor flexibility; (2) there is duplication and redundancy across the laboratories; and (3) the diversification of the facilities into new activities lacks focus, particularly after funding for the core nuclear activities declined. As noted by a recent synthesis of these studies by the U.S. General Accounting Office (1998, p. 15), "despite many studies identifying similar deficiencies in the management of DOE's laboratories, fundamental change remains an elusive goal."
[] Patent policy and the DOE FFRDCs. Prior to 1980, the laboratory contractors were assigned few patents and exclusive licenses were rare (see Table 2).(7) Although DOE policy began to change after the legislative changes of the 1980s, a number of accounts (e.g., U.S. Office of Technology Assessment, 1993) suggest that DOE's response was delayed until the period between 1986 and 1989.
TABLE 2 R&D Spending and Technology Transfer at DOE FFRDCs
Energy
Agency-Funded Total R&D DOE
R&D at DOE at DOE Licensing
FFRDCs FFRDCs Revenues
Fiscal (billions of (billions of (millions of
Year 1995 dollars) 1995 dollars) 1995 dollars)
1955 1.0
1956 1.6
1957 2.0
1958 2.2
1959 2.5
1960 2.7
1961 2.9
1962 3.0
1963 3.2
1964 3.7
1965 3.6
1966 3.4
1967 3.5
1968 3.6
1969 3.5
1970 3.4
1971 3.2
1972 3.2
1973 3.3
1974 3.1
1975 3.4
1976 3.5
1977 4.5
1978 4.7
1979 4.7
1980 4.5
1981 4.5 5.0
1982 4.2 4.9
1983 4.3 4.8
1984 4.5 5.0
1985 4.8 5.3
1986 4.5 5.1
1987 4.4 5.6 .4
1988 4.4 5.6 .7
1989 4.6 5.5 1.8
1990 4.4 5.4 2.9
1991 4.3 5.2 3.5
1992 4.3 5.1 2.5
1993 3.7 4.3 2.8
1994 3.4 3.7 3.0
1995 3.3 3.6 3.5
1996 3.1 3.2 4.0
1997 3.1 3.4 5.5
Invention Exclusive Nonexclusive Active
Disclosures Licenses Licenses CRADAs
from DOE from DOE from DOE at DOE
Fiscal FFRDCs FFRDCs FFRDCs FFRDCs
Year (number) (number) (number) (number)
1955
1956
1957
1958
1959
1960
1961
1962
1963 1,627
1964 1,724
1965 1,649
1966 1,271
1967 1,257 0 28
1968 1,138 0 41
1969 1,235 0 85
1970 1,420 0 127
1971 1,502 0 102
1972 1,129 0 66
1973 1,228 0 56
1974 1,170 0 38
1975 1,125 0 70
1976 1,533 1 31
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987 857 14 23
1988 1,003 18 25
1989 1,053 25 32
1990 1,335 30 58 1
1991 1,665 29 96 43
1992 1,698 24 191 250
1993 1,443 30 378 582
1994 1,588 37 470 1,094
1995 1,758 61 575 1,382
1996 1,886 82 662 1,677
1997 68 189 749
Many observers also suggest that DOE's implementation of the 1980s reforms was problematic. Critics have attributed this to two factors:
First, the problems with the laboratory structure in general noted above led to resistance to these reforms. One illustration is the process through which proposed cooperative agreements with industry were evaluated. Although in many agencies laboratory directors were allowed to implement CRADAs with limited headquarters oversight or regulatory requirements, DOE introduced a three-part review process. CRADA proposals were reviewed at the laboratory level, followed by a centralized screening by the headquarters program offices (typically on an annual basis), followed by the negotiation of a contract and a work statement (which involved laboratory, field office, and program office personnel). Not surprisingly, during the period when DOE was actively seeking new CRADAs, the level of activity was lower than in other agencies and the time from inception to signing much longer (U.S. General Accounting Office, 1993).
Second, the fundamental conflict between the commercial need for clear private property rights in the form of exclusive licenses and the broader goals of public research seemed particularly acute at these facilities. In many cases, spin-out firms or cooperative partners were not able to obtain exclusive licenses to DOE patents, or were able to do so only after delays of many years. Some licenses were subsequently challenged as violations of conflict-of-interest or fairness-of-opportunity rules in lawsuits by competitors or in congressional hearings (see Markusen and Oden, 1996). DOE radically scaled back its CRADA program in 1995 and 1996 after Republican congressmen questioned whether it represented "corporate welfare" (Lawler, 1996).
These difficulties had a negative impact on corporations' or venture capitalists' willingness to invest in these projects. Reviews of technology-transfer activities at the national laboratories have almost universally highlighted the lack of progress, particularly when contrasted to universities.(8) This appears to have been a general problem that affected technology transfer at all the laboratories.
3. Two case examples
* We briefly illustrate these challenges through a discussion of two laboratories, Lawrence Livermore National Laboratory (LLNL) and the Idaho National Engineering and Environmental Laboratory (INEEL). These facilities are in some respects very different. The former has DOE's Office of Defense Programs as its primary funder, has an academic institution as a prime contractor, and is located in the San Francisco Bay Area. The latter is funded primarily by DOE's Office of Environmental Management, has a private corporation as the prime contractor, and is located in a remote area of eastern Idaho. But both facilities have overcome considerable challenges to develop technology-transfer efforts that in many respects represent "best practice" among the DOE FFRDCs. LLNL and INEEL have led the laboratories in the level of licensing revenue and spin-out companies respectively.
While a variety of insights can be drawn from the analysis, three observations were particularly salient for this analysis:
(i) In contrast to the pessimistic conclusions of many studies, the DOE laboratories have the potential for real commercial impact. LLNL has a long tradition of informal collaboration with industry. The reforms at INEEL appear to have triggered a wave of spin-out activity commercializing technology at the laboratory.
(ii) The uncertainty associated with the ownership of laboratory technologies is a barrier to commercialization. Unlike universities, where the Bayh-Dole Act of 1980 created a clear understanding that universities could license their technology, for the laboratories the property rights remain unclear. Despite the reforms of the 1980s, substantial questions remain about the ability of laboratory contractors to freely license their technologies.
(iii) The reforms of technology-transfer practices have had a dramatic impact on commercialization activities. At INEEL, a change in the managing contractors and the adoption of an innovative incentive contract appear to have had a dramatic effect on technology transfer. At Livermore, although neither the policy shifts nor the alteration in the volume of technology-transfer activity were as dramatic, shifts in government patent policy appear to have had a substantial impact on the pace of activity.
[] Lawrence Livermore National Laboratory. Lawrence Livermore is one of the three very large DOE FFRDCs that has historically specialized in nuclear weapons research. LLNL was established in 1952 in Livermore, California. The University of California has operated this facility from its inception. From the first, LLNL had a strong emphasis not only on the engineering of thermonuclear weapons, but on related fundamental science. It was an aggressive user of the newest computational technology: for instance, the first facility at Lawrence Livermore was a building to house the then-new UNIVAC computer, which was needed for the complex calculations required by the weapons design process. As at other facilities, there has been a broadening of the laboratory's mission over time into such areas as nonnuclear energy, biomedicine, and environmental science.
The evolution of the technology-transfer function at the laboratory has similarly featured both continuity and change. At least since the 1960s, the laboratory has had strong relationships with the computer and laser industries. Advances in the state of the art developed at the laboratory were often transferred to private firms or developed by companies in response to laboratory requests, in order to generate a production-scale source of equipment, instrumentation, or components for the larger experimental facilities. In many cases, the prototypes were cooperatively developed by private-sector and laboratory researchers. A significant number of these innovations eventually found their way into the civilian market. The laboratory's motivation for engaging in this activity, however, had little to do with concerns about "technology transfer." Rather, the staff went to the private sector because it was often more efficient to procure equipment from outside vendors than to manufacture it at the laboratory. The relationships with vendors were highly informal. LLNL made virtually no effort to claim intellectual property holdings, and in many cases their partners did not seek to patent the discoveries either.
A formal technology-transfer office was established at LLNL after the DOE issued the implementing regulations for the Stevenson-Wydler Act in 1982. Initially the office was modestly funded, with little internal or external visibility. There was a dramatic increase in activity, however, after the passage of the National Competitiveness Technology Transfer Act of 1989. In particular, the DOE established a central program (later known as the Technology Transfer Initiative) primarily to fund CRADAs between the laboratories and companies. With this influx of funding--in 1994 alone, LLNL received $55 million for this purpose from the DOE--the technology-transfer office (now known as the Industrial Partnerships and Commercialization (IPAC) office) grew rapidly. During this relatively brief era, LLNL collaborated with industry partners to carry out almost 200 mostly small, jointly funded projects spanning a broad spectrum of technology areas. A great deal of effort was devoted to writing proposals, attending trade shows, and hosting visiting delegations.
This period left two key legacies. First, the laboratory established a few significant relationships, such as with semiconductor manufacturers, that would have ongoing importance. Second, an infrastructure was developed to better interact with industry, i.e., the ability to enter fairly quickly into agreements, to protect proprietary information, and to allocate the intellectual property generated in the agreements. As time progressed, however, laboratory officials felt that such a large number of partnerships in relatively unfocused technology areas were distracting personnel from the facility's primary programs.
In 1995, the U.S. Congress, whose control had just switched to the Republican Party, dramatically cut funding for the Technology Transfer Initiative. The DOE followed suit with additional cuts. This triggered a shift at LLNL back to its original focus on just undertaking industrial partnerships related to its mission. Projects with outside companies that were only tangentially related to the laboratory's mission were largely terminated. As a result, LLNL as of November 1998 had a few very large technology-transfer efforts, a vastly reduced number of smaller R&D projects, and a growing number of licenses. At one extreme was the CRADA at LLNL and two other laboratories to produce semiconductors through extreme ultraviolet lithography, to which Advanced Micro Devices, Intel, and Motorola were contributing $250 million over three years. At the other extreme were numerous licenses of laboratory technology, primarily with small high-technology firms. (See panel A of Table 3 for a summary of LLNL's technology-transfer activities.)
TABLE 3 Technology-Transfer Activity at Two DOE FFRDCs Panel A: Lawrence Livermore National Laboratory Fiscal Year 1989 1990 1991 1992 1993 Invention disclosures filed 141 192 193 270 219 Patent applications filed 29 68 60 43 92 Patents received 37 44 48 60 New patent licenses 3 6 8 4 Licensing revenues (thousands of 1995 dollars) 313 462 437 373 New CRADAs formed 18 57 Startups formed from laboratory technology Fiscal Year 1994 1995 1996 1997 Invention disclosures filed 264 205 240 193 Patent applications filed 89 126 130 134 Patents received 75 90 100 71 New patent licenses 11 28 47 28 Licensing revenues (thousands of 1,118 1,310 2,067 1995 dollars) 575 New CRADAs formed 60 67 34 37 Startups formed from laboratory technology 0 Panel B: Idaho National Engineering and Environmental Laboratory Fiscal Year 1992 1993 1994 1995 1996 1997 1998 Patent applications filed 11 14 24 38 46 57 61 New patent licenses 5 9 12 36 23 18 Licensing revenues (thousands of 1995 dollars) 7 16 97 86 207 343 386 Startups formed from laboratory technology 0 1 0 5 10 7 5
Lawrence Livermore's success in licensing relative to the other laboratories had been facilitated by its strong ties to the University of California, as well as its physical proximity to the companies and financiers in the Bay Area. The IPAC office frequently interacted informally with the licensing staff at the University's Office of Technology Transfer. At the same time, the office faced challenges that its university-based peers did not. In particular, the IPAC office did not automatically receive title to patents (as universities have since the passage of Bayh-Dole). Rather, the staff had to formally request waivers from the DOE on a case-by-case basis, which could be a lengthy process. Second, the office faced much greater scrutiny of its actions under rules governing fairness of opportunity and conflict of interest. As a result of this scrutiny, the IPAC extensively publicized potential licensing opportunities and took great care to avoid these problems, including often encouraging prospective licensees to accept nonexclusive licenses.
LLNL's licensing activities and revenues rose dramatically in the 1990s. But it faced a continuing challenge posed by its diffuse and changing mandate from the U.S. Congress. For instance, the directives to the laboratories to transfer technology in a way that benefits the U.S. economy, to ensure fairness of opportunity, and to avoid competition with the private sector were interpreted very differently by various members of Congress. Anticipating how these concerns would evolve over time, and which transactions might be seen as problematic in retrospect, was not easy. A compounding factor was that as the laboratory's technology-transfer effort became more visible, it was increasingly a target of complaints and scrutiny, as illustrated by the case described in footnote 3.
[] Idaho National Engineering and Environmental Laboratory. Shortly after World War II, the federal government sought an isolated location to test nuclear reactors. The predecessor to INEEL, the National Reactor Testing Station, was established in 1949 on an 890-square-mile site in the southeastern Idaho desert that had been used as a practice bombing range during World War II. Over the years, the laboratory undertook major research programs seeking to develop prototypes of and to test reactors for both naval vessels and (until 1961) airplanes. Another important activity was the reprocessing of the large amounts of uranium generated by these reactors. Unlike LLNL, which from its inception has had only one contractor, INEEL was managed over the years by a series of contractors. These included Aerojet Nuclear (a subsidiary of General Tire) and a consortium including EG&G and Westinghouse Electric (between 1977 and 1994).
As the Management and Operation Contract for the laboratory neared completion in 1994, it became clear that a major emphasis in selecting the next contractor would be a commitment to technology-transfer activities. INEEL was a major employer in the state of Idaho. Local politicians had argued that more efforts were needed to soften the impacts of employment and funding cutbacks at the laboratory by encouraging the creation of spin-out firms. The minimal level of technology-transfer activity in this period can be seen in panel B of Table 3: for instance, in fiscal year 1992, there were no spin-outs and the laboratory generated only $7,000 in licensing revenue.(9) While EG&G rapidly increased its signing of CRADA and licensing agreements in response to these pressures, the new contract was awarded instead to a consortium led by a subsidiary of the Lockheed Corporation (the company's name was changed to Lockheed Martin in 1995). Among the participants in the consortium was Thermo Electron Corporation, a Massachusetts company with a history of spinning out new technology businesses into publicly traded entities.
The contract included a variety of features to help ensure that technology-transfer activities would be taken seriously. The contractors committed to provide entrepreneurial training to laboratory researchers who were prospective leaders of spin-out firms. Thermo Electron committed to establish a $10 million venture capital fund to be made available to finance new businesses spinning out from the laboratory. Perhaps most important, Lockheed signed a contract under which its reward would be a function of its technology-transfer activity. In particular, Lockheed agreed to forgo several millions of dollars from its annual fee for managing the laboratory. In return, it received a share of fees and royalties, which increased with the cumulative amount of payments received over the course of the five-year contract. Until the first $1 million of licensing payments was received, the firm would receive 20% of the revenue (the remainder being divided between the researcher and the federal government). For the next $1 million, Lockheed would receive 30%, and thereafter it would receive 35%.(10)
To implement this contract, the new contractor undertook a variety of changes to the structure of the technology-licensing office as well. Lockheed recruited individuals who had held senior business development positions with companies such as General Motors and IBM, as well as licensing account executives with private-sector sales and marketing experience. In addition, the company organized industry focus teams, with the responsibility to establish relationships with and market INEEL capabilities and technologies to companies in specific industries. As seen in panel B of Table 3, both spin-out and licensing activity increased dramatically in response to these activities. The increase in spin-out activity was particularly noteworthy: in fiscal year 1997, INEEL accounted for 7 out of the 19 spin-outs from DOE FFRDCs.
At the same time, the implementation of this plan faced unexpected difficulties. One was the extent of the barriers to spinning out of technologies from INEEL. Laboratory researchers seeking to obtain an exclusive license to a technology they had worked on faced an exhaustive and slow review process. Even when the entrepreneurs overcame concerns about fairness and conflict-of-interest provisions, in some instances they believed that the laboratory's demands for payments and royalties were excessive, given the early stage of the technologies. Some felt that the management contract gave the contractor incentives to license technologies to large corporations that could offer larger upfront payments than startups could afford. As of the end of fiscal year 1997, Lockheed had received a total of about $130,000 from its share of the royalties, only a few percent of the amount forgone in fees. At the same time, it is important to acknowledge the dilemma that Lockheed officials faced when considering licensing to prospective startups. To assess whether a startup was the entity with the greatest chance of commercializing an INEEL technology, the technology-transfer officials typically asked it to provide a business plan and show proof of adequate financing. But in many cases, entrepreneurs found (particularly in a state like Idaho, where little financing of small high-technology firms was available) that obtaining a license to the technology was a prerequisite to be considered for financing. Another challenge was the difficulty of identifying people with the entrepreneurial business talents to complement the laboratory scientists and engineers in a spin-out. In addition, entrepreneurs faced barriers to raising financing once they had exhausted the seed capital that the small Thermo Electron fund and other local investors could provide.
Despite the considerable success of the INEEL effort, its future was in doubt at the end of 1998. The DOE decided in September 1998 not to renew Lockheed's contract, but rather to put the contract once again up for bid. While the DOE review rated Lockheed's technology-transfer effort highly, the agency raised concerns about the contractor's record in worker safety and its failure to undertake an environmental cleanup project at INEEL for an agreed-upon price. While the DOE made it clear that it expects the next contractor to be committed to technology transfer, whether that contractor will be as successful as Lockheed in growing those activities remains uncertain.
4. Statistical analysis
* The dataset. A point raised in many assessments of the national laboratory system (e.g., Task Force (1995), Section VII.C.5) is the extreme difficulty in obtaining data, particularly with regard to technology-transfer activities. This is at least partially a reflection of the DOE's complex management structure discussed above. As a result, we have constructed a dataset from a wide variety of sources.
We constructed a sample of 23 FFRDCs owned by the DOE and active between 1977 and 1997, derived from Burke and Gray (1995)(11) and U.S. National Science Foundation (various years). Two of these organizations commenced operations as FFRDCs during this period, while six were decertified for various reasons.
General information. Information collected for each facility included:
(i) Historical information such as dates of establishment or decertification, and identity of contractor over time (Burke and Gray, 1995; U.S. General Accounting Office, 1996b; facility Web sites; news stories in LEXIS/NEXIS).
(ii) Regional characteristics such as the distance to the nearest standard or consolidated metropolitan statistical area (SMSA or CMSA), the population and education level of that area, and venture capital activity in the state (U.S. Bureau of the Census; Venture Economics' Venture Intelligence Database (described in Gompers and Lerner, 1999)).
(iii) Overall laboratory budget level and funding sources.(12)
(iv) Annual R&D expenditures at each FFRDC between 1981 and 1995 (National Science Board, 1996; U.S. National Science Foundation, 1998).
(v) The number of new CRADAs formed annually between 1991 and 1994 under the aegis of three program offices (Defense Programs, Energy Efficiency, and Energy Research), and all CRADAs formed in 1995 and 1997.(13)
(vi) Miscellaneous data about a variety of technology-transfer activities (U.S. Department of Energy, Office of Defense Programs, 1998; U.S. Department of Energy, Office of the General Counsel, 1998).(14)
Table 4 provides summary data on these facilities.
TABLE 4 DOE FFRDC Cross-Sectional Data
1997 Licenses
Contractor Revenue
FFRDC Type Number ($M)
Ames Univ 5 .005
Argonne Univ 13 .134
Bettis Atomic Power Firm
Brookhaven Univ 40 1.342
Lawrence Berkeley Univ 56 .351
Lawrence Livermore Univ 19 2.118
Energy Technology Engineering
Center Firm
Fermi Accelerator Univ 0 .001
Hanford Engineering
Development Firm
Idaho Engineering and
Environment Firm 16 .358
Inhalation Toxicology Other
Knolls Atomic Power Firm
Los Alamos Univ 4 .379
Mound Firm
Renewable Energy Research Other
Oak Ridge National Laboratory Firm 38 1.289
Oak Ridge Institute for Science
and Education Univ
Pacific Northwest Other
Princeton Plasma Physics Univ 1
Sandia Firm 59 1.3
Savannah River Firm 5 .021
Stanford Linear Accelerator Univ 3 .2
Thomas Jefferson Accelerator Univ
Unattributed
1995 R&D Shares
Patent
Class National Basic
FFRDC Concentration Security Science
Ames .15 .0% 79.0%
Argonne .11 3.3% 51.8%
Bettis Atomic Power .15 100.0% .0%
Brookhaven .11 5.0% 91.0%
Lawrence Berkeley .11 .0% 78.0%
Lawrence Livermore .08 61.6% 16.0%
Energy Technology Engineering
Center .12
Fermi Accelerator .13 .0% 100.0%
Hanford Engineering
Development .38
Idaho Engineering and
Environment .14 12.0% 28.0%
Inhalation Toxicology .33
Knolls Atomic Power .10 100.0% .0%
Los Alamos .08 76.0% 13.0%
Mound .09
Renewable Energy Research .09 .0% 2.0%
Oak Ridge National Laboratory .09 5.0% 42.0%
Oak Ridge Institute for Science
and Education .22 9.0% 64.0%
Pacific Northwest .11 13.0% 14.0%
Princeton Plasma Physics .51 .0% 100.0%
Sandia .08 77.0% 3.0%
Savannah River .17 18.5% .0%
Stanford Linear Accelerator .21 .0% 100.0%
Thomas Jefferson Accelerator .20 .0% 100.0%
Unattributed
FY 1997
Licenses
Royal
State ties
Share of No. ($M)
Venture Contract per per
Capital Competitively $MM $MM
FFRDC (1988) Awarded R&D R&D
Ames .61% No .24 .24
Argonne 1.78% No .05 .55
Bettis Atomic Power 4.05% Yes
Brookhaven 3.75% No .18 6.04
Lawrence Berkeley 31.51% No .24 1.52
Lawrence Livermore 31.51% No .03 3.76
Energy Technology Engineering
Center 31.51% Yes
Fermi Accelerator 1.78% No .00 .01
Hanford Engineering
Development 1.84% Yes
Idaho Engineering and
Environment .06% Yes .25 5.70
Inhalation Toxicology .37% Yes
Knolls Atomic Power 3.75% No
Los Alamos .37% No .01 .67
Mound 2.21% Yes
Renewable Energy Research 3.13% Yes
Oak Ridge National Laboratory 1.78% Yes .16 5.51
Oak Ridge Institute for Science
and Education 1.78% No
Pacific Northwest 1.84% No
Princeton Plasma Physics 3.50% No .02 .00
Sandia .37% Yes .09 1.98
Savannah River .18% Yes .29 1.22
Stanford Linear Accelerator 31.51% No .03 1.69
Thomas Jefferson Accelerator 2.09% Yes
Unattributed
Successful
Patent Overall
Applications Citations
per
FFRDC 1977 1993 Patent CRA-DAs
Ames 0 11 1.77 10
Argonne 0 32 1.27 111
Bettis Atomic Power 9 13 2.65 0
Brookhaven 6 15 2.39 40
Lawrence Berkeley 0 14 1.38 140
Lawrence Livermore 26 103 2.44 237
Energy Technology Engineering
Center 4 1 4.20 0
Fermi Accelerator 2 1 2.16 0
Hanford Engineering
Development 0 1 .25 0
Idaho Engineering and
Environment 0 15 .50 55
Inhalation Toxicology 0 0 3.00 3
Knolls Atomic Power 3 1 2.00 0
Los Alamos 11 20 2.94 238
Mound 1 5 1.43 NA
Renewable Energy Research 1 19 1.68 41
Oak Ridge National Laboratory 0 37 1.50 238
Oak Ridge Institute for Science
and Education 0 1 .17 1
Pacific Northwest 0 10 1.60 54
Princeton Plasma Physics 0 4 1.16 4
Sandia 0 34 .88 260
Savannah River 1 32 .96 17
Stanford Linear Accelerator 1 12 2.04 2
Thomas Jefferson Accelerator 0 2 .36 0
Unattributed 100 73 3.47
Measures of patenting activity. The final set of variables measures the number of and citations to patents derived from laboratory research. This subsection reviews the difficulties associated with measuring patenting activity at the FFRDCs, the DOE database that allows us to address these problems, and the issues associated with the use of the database.
Patent awards from DOE research are sometimes assigned to the department; at other times when waivers are granted, they are assigned to the contractors. There is no single identifier in the patent application that allows one to identify DOE-funded patents: e.g., it is difficult to distinguish from the text of a patent assigned to the University of California whether it was derived from work at the university's Berkeley campus or at Lawrence Livermore. To address this problem, we employ a database compiled by the DOE's Office of Scientific and Technical Information (1998) of all patents to emerge from DOE laboratories since 1978.
The database, which consists of 6,479 U.S. patents awarded by the end of 1996, contains all patents arising from laboratory-produced research, regardless of the entity to which the patent is assigned. In particular, it contains patents assigned to the contractor who operates a given laboratory. U.S. Patent and Trademark Office databases (as well as the NBER/Case Western Reserve database discussed below) identify patents assigned to the U.S. government and its agencies. They do not provide, however, any way to separate patents assigned to firms in their capacity as contractors operating government laboratories from those derived from the contractor's other research (see Jaffe, Fogarty, and Banks, 1998). Of the patents we have included in our analysis, 42% were assigned to nongovernmental entities.
The file also has two major drawbacks. First, it contains awards assigned to entities that have never operated a DOE FFRDC. It is likely that some of these are patents derived from CRADAs between a laboratory and another entity. Others are derived from other contractor-operated facilities that are not designated as FFRDCs. Some of the patents, however, are apparently not derived from DOE-funded research but are merely in an area of interest to the department. A second difficulty is that the database does not provide any direct means to connect the patents to particular FFRDCs. Most, though not all, of the patents in the database are identified with a contract number that corresponds to the Management and Operation contract at the originating facility at the time of the patent award. These are identified through a database maintained by DOE's Office of Procurement and Assistance Management (1998) as well as personal communication with officials of this office.
The list of DOE patents was then merged with the NBER/Case Western Reserve patent database. This allows us to determine the patent class, application year, and the address of the inventor for each patent. (We also identified all patents that cited these patents through the end of 1995.) Given these ambiguities, we followed the following procedure:
First, every patent whose contract number in the database corresponds to a known Management and Operation contract was attributed to that FFRDC.
Second, every patent without a contract number whose assignee was the DOE and whose primary inventor lived in a SMSA where there was a DOE FFRDC was assumed to come from that laboratory.
Third, every patent without a contract number whose assignee was a laboratory contractor, and whose primary inventor lived in a SMSA where there was a DOE FFRDC run by that contractor, was assumed to come from that FFRDC.
Fourth, patents that could not be attributed to a specific FFRDC, but which most likely derived from DOE laboratory research, were used for aggregate analysis but were not included in the laboratory-specific analyses. These include patents with unidentified contract numbers, patents assigned by the patent office to DOE with inventors residing in SMSAs with multiple laboratories, and patents assigned to contractors with multiple laboratories in a given SMSA, with inventors residing in that SMSA.(15)
All other patents in the database were not used in the analysis below.
Overall, of the 6,479 patents in the DOE database, 3,185 were attributed to particular DOE FFRDCs, 1,771 were determined to be laboratory patents but were not attributed to any particular facility,(16) and 1,523 could not be determined to be derived from laboratory research and hence were ignored. Our approach undoubtedly both includes some nonlaboratory patents and excludes some laboratory patents. Overall, we are probably undercounting laboratory patents. Many of the 1,523 ignored patents have inventors living in a SMSA in which there is a laboratory, but they are assigned to firms other than the contractor. It is likely that some of these derive from CRADAs, but we have no way to determine that and so have excluded them.
Throughout this article, we date patents by the year of application, since that is when the research was likely to have occurred and the decision made to apply for the patent. The DOE patents database contains only those patents awarded through the end of 1996. Since it typically takes between one and two years to process a patent application, we seriously undercount applications made in the years after 1993.
Table 4 shows the patent totals for each of the 23 DOE FFRDCs. Many of the laboratories had no successful patent applications in 1977. By the end of the period, most made successful patent applications every year. (Note that four laboratories were decertified by 1992.) Not surprisingly, a handful of the larger laboratories contribute most of the patent applications filed in each year.
[] Analyses of overall technology commercialization patterns. As discussed above, the numerous statutory changes in the 1980s were intended to foster the commercialization of federally funded R&D. If successful, these changes should have increased the rate of patenting of laboratory discoveries and the utilization of these inventions by the private sector. To explore this question, we look at the time trend in patenting and the citations to these patents, as well as other measures.
Figure 2 shows that the number of successful DOE laboratory patent applications rose from about 200 in 1981 to over 450 by 1993. (As noted above, the latter number is a slight underestimate, because some applications will have been granted after 1996.) Furthermore, this sharp increase in patenting has occurred in the face of declining real R&D expenditures at the DOE FFRDCs. As shown in the figure, the increase began in 1988, shortly after the passage of the Federal Technology Transfer Act, and continued into the mid-1990s.
[Figure 2 ILLUSTRATION OMITTED]
To put this increase in the "propensity to patent" in perspective, Figure 3 compares the patent-to-R&D ratio for the DOE FFRDCs to a similar ratio for universities.(17) In the early 1980s, patents per dollar of FFRDC R&D were considerably lower than in the universities, despite the fact that two-thirds of the university research was for basic research, which is presumably less likely to lead to patent awards. (The comparable figure for all federal FFRDCs is slightly over 40%. The source of these tabulations is National Science Board (1998).) University patenting rose strongly through the 1980s, at least partially in response to the Bayh-Dole Act of 1980. But by the late 1980s, patenting (relative to R&D spending) at the DOE FFRDCs was rising even faster. By 1993, the two sectors were comparable in terms of patents per R&D dollar. While the lower share of basic research at the laboratories makes this a somewhat unfair comparison, the relative performance of the DOE FFRDCs in the late 1980s and early 1990s was nonetheless remarkable.(18)
[Figure 3 ILLUSTRATION OMITTED]
It has been shown that the dramatic increase in university patenting in the 1980s was accompanied by an equally sharp decline in the quality of those patents, as measured by the citations they received relative to all patents with the same technological characteristics and award year (Henderson, Jaffe, and Trajtenberg, 1998). Before about 1985, university patents on average were much more highly cited than other patents. This difference had disappeared by the late 1980s. The increase in university patenting appears to have come about largely by "lowering the threshold" for patent applications, resulting in many more patents of marginal significance.
To explore whether similar changes occurred at the DOE FFRDCs, we construct the "normalized" citation intensity for each laboratory in each year. To do this, we calculate the difference between the actual number of citations received per patent and the "reference" citation intensity. The reference citation intensity is the expected number of citations per patent that a portfolio of patents with the same technological classifications as those of the laboratory would receive, based on the citations received by all patents in a given technology class in a given year. This normalized intensity controls for differences across technology classes and time in the "propensity to cite," as well as for the impact of the truncation imposed by our lack of knowledge of citations that will occur after 1995.
We also report in Figure 4 the trend in patenting. Here we employ not the aggregate count of patents derived from DOE FFRDCs, but rather a normalized series that controls for the shift in the overall "propensity to patent." In each technology class and in each year, we compute the ratio of the number of patents derived from the DOE laboratories to the total number of awards. The changes in this value, which is normalized to be 1.0 in 1981, are plotted over time. During periods when the overall number of patent applications is falling, this normalization will make an increase in DOE patenting appear more dramatic, and vice versa.
[Figure 4 ILLUSTRATION OMITTED]
Figure 4 shows a citation pattern in conjunction with increased patenting for laboratories that is very different from that found for universities. First, note that in contrast to the university patents, laboratory patents have historically been slightly less highly cited than other patents.(19) It is striking, however, that the laboratory citation intensity did not decline in conjunction with the large increase in the propensity to patent after 1987. Whether we look at a simple average across the laboratories or an average weighted by the number of patents, the trend in citation intensity is upward. This trend, however, is not statistically significant. Thus, the kind of "digging deeper into the barrel" that characterized the increase in university patenting does not seem to have occurred at the DOE FFRDCs. While this clearly merits more study, the citation data are consistent with a process in which the laboratories produced more patents by reorienting their research toward new areas with greater commercial applicability, in keeping with the intent of the statutory changes.
The pattern is also similar if we look only at "non-self-citations." Non-self-citations exclude those citations in patents that are assigned either to the DOE or to the contractor that operates the laboratory from which the cited patent originated. The non-self-citation measure is "normalized" as well, following a procedure similar to that described above but only using non-self-citations to construct the reference portfolio. The fact that the normalized non-self-citation pattern over time is so close to the normalized total citation pattern means that the rate of self-citation to laboratory patents is similar, on average, to the rate of self-citation for all patents.
It would also be desirable to look at other indicators of technology transfer. Unfortunately, we have only extremely limited and inconsistent data on other indicators of technology transfer and very few measures, particularly over time, of laboratory policies and behaviors. Below we explore other measures of technology transfer in panel and cross-sectional analyses of the laboratories. Before turning to that, however, we undertake panel data regression analyses using patents and citations.
[] Cross-sectional analyses using patent data. We now turn to examining the differences across facilities. But before doing so, it makes sense to catalog the theoretical suggestions about which factors should affect the relative success of such a policy shift across different laboratory environments.
First, some kinds of research are inherently more difficult to commercialize. Laboratories that are particularly tied to national security issues or devoted to fundamental scientific research are less likely to have commercialization opportunities.
Second, laboratories that have pursued unfocused diversification efforts may have lower-quality research; this echoes the literature on adverse effects of diversification in the corporate setting (Lang and Stultz, 1994; Scharfstein and Stein, 1997). If diversification reduces research quality, it would lead to less "product" to transfer.
Third, organizational factors are also likely to matter. As discussed above, the 1980 Bayh-Dole reforms gave a great degree of flexibility to universities to license and spin out new technologies. Many academic institutions exploited these changes by building up technology-licensing offices and aggressively marketing new technologies (Henderson, Jaffe, and Trajtenberg, 1998). FFRDCs whose prime contractors were universities may have benefited from this know-how to make their technology transfer activities more effective.
Fourth, the nature of the contractor and its relationship with the DOE might have had an important influence. It might be thought that political direction was more difficult at organizations where the prime contractor had a long-standing presence at the laboratory. Efforts to encourage exclusive licensing have been highly controversial within the DOE. A particular source of resistance has been the DOE field offices, where in many cases the staff has questioned the desirability of senior management's efforts to change technology-transfer policies. (One motivation for these concerns may be the loss of control inherent in the delegation of a great deal of discretion to contractors' technology-licensing offices.) A new contractor typically brings in new staff when it receives an award to run a facility. Over time, however, this staff may develop close working relationships with DOE officials in the local field office. Consequently, resistance to residual efforts to assert control by DOE staff may be the greatest in a setting where there is a new contractor.(20)
Of course, it may be that poor performance on the part of the laboratory contributes in turn to the probability of contractor turnover. An example of this type of "reverse causality" may be the INEEL case discussed above: EG&G's poor performance in technology transfer appears to have been a contributing factor in its loss of the management contract. In other cases, the replacement of the contractor appears to have been driven by outside events, and to have been largely exogenous. An example was AT&T's decision to cease managing Sandia in 1992, after the divestiture of its regional operating units and the scaling back of much of the basic research at Bell Laboratories.
Finally, one might expect that geographic location would affect the success of technology transfer. Recent work has shown that knowledge spillovers tend to be geographically localized, particularly within geographic areas (Glaeser, et al., 1992; Jaffe, Trajtenberg, and Henderson, 1993). This concentration of knowledge spillovers has implications for the relative level of technology transfer at the national laboratories because these facilities are located in very different areas. Some facilities are located far from any major metropolitan area, while others are near a major metropolis. If it is easier to transfer technologies to nearby firms, the laboratories located in remote areas will have more trouble finding recipients. This disadvantage could be amplified by the concentration of venture capital, the primary mechanism for funding for privately held, high-technology companies. Venture organizations are highly geographically concentrated, with over half the funds based on California and Massachusetts. (Gompers and Lerner (1999) provide a detailed description.) Furthermore, venture capitalists tend to be highly localized in their investment patterns: over half the venture-backed firms have a venture investor who serves as a board member based within 60 miles of the firm (Lerner, 1995). Thus, even if laboratories in remote areas with little venture capital activity do generate spillovers to nearby companies, the local firms may be unable to access the needed financial and other resources to profit from them.(21)
The regression analysis covers the period from 1981 to 1993. The first year of the analysis is determined by the availability of R&D spending data. The final year of the analysis is determined by the problem of lengthy patent-pending periods discussed above.
As dependent variables, we employ either the count of patent applications or the citations per patents at each DOE FFRDC in each year. Both the count of patents and the citations per patents are normalized to control for differences across technological classes, as described above. Because we believe that the "patent production function" at the laboratories will be multiplicative rather than additive--e.g., the policy shifts of the 1980s should have led to a more dramatic absolute increase in patenting at the larger laboratories--we employ the logarithm of normalized patenting as the dependent variable.(22) We employ both normalized total citations per patent and the normalized ratio excluding self-citations as dependent variables.
We employ a variety of independent variables. To capture the shifting regulatory environment, we employ a dummy variable that denotes whether the annual observation is from 1987 or after. This is roughly when numerous accounts suggest that there was a substantial shift in how seriously the DOE took its mandate to implement technology transfer. Second, we identify the periods when the contractor was changed. As suggested by the Idaho case study, such changes may provide the stimulus to focus real effort on objectives such as technology transfer. We arbitrarily hypothesize that the effect of such changes--both the dislodging of bureaucratic inertia and the incentive to improve performance--might be seen from two years before the change through two years after. There is a total of fifteen lab-years in the data that fall into such windows, about 8% of the data points used in the regression.
We also include in the regression a few characteristics of the laboratory or its environment. We employ the R&D of the lab in the patent equation, and the R&D and patenting rates in the citation equation. These variables control for the scale of the research enterprise, and they also allow the citation intensity to be different as the rate of patenting relative to R&D effort changes.
We also include a variety of time-invariant measures. First, as discussed above, a strong orientation toward national security or basic science may lead to less technology commercialization. We thus include as independent variables the shares of the laboratories' expenditures classified as related to national security and basic science. (Due to data limitations, we use 1995 values, but these measures appear to be quite constant.) Second, we indicate whether the contractor was a university, and likely to be more familiar with the transfer of early-stage technologies, by employing a dummy variable that assumes the value of unity when this was the case. Third, many reports have claimed that the laboratories' efforts to diversify away from their traditional areas of expertise have led to poor performance. To examine this suggestion, we construct from the patent data a measure of technological "focus": the Herfindahl index of concentration of patenting across technology classes.(23) We include both the measure of focus as well as the change in our focus measure between the second and first half of the sample period.(24)
Estimation results are presented in Table 5. Columns 1-3 present the results of the patent regressions, and columns 4-7 are for citation intensity. Our data are in the form of a panel, in which the patenting or citation intensity for a given lab in each year depends upon lab variables that change over time, observable lab characteristics that do not vary over time, and (most likely) unobservable lab characteristics that are also correlated with both the time-varying regressors and the observable time-invariant lab characteristics. In this context, OLS on the pooled panel data produces potentially biased estimates, because of the correlation of the unobserved lab effect with the regressors. A "fixed effects" regression, or, equivalently, a regression including a set of lab dummy variables, yields unbiased estimates of the effect of the time-varying regressors.
TABLE 5 Panel Regression Results--Patents and Citations
Dependent Variable
Logarithm of Logarithm of
Patents(a) Patents(a)
Independent Variables (1) (2)
Year = 1987 or later .515 .514
(***)(.098) (***)(.081)
Logarithm of R&D .223 -.139
in fiscal year (***)(.089) (.181)
Logarithm of patents(a)
"Competition"(d) .019 .172
(.222) (.229)
"Focus"(e) 1.471
(**)(.598)
Change in "focus"(e) 1.804
(***)(.331)
National security share of R&D -.192
(FY 1995) (.202)
Basic science share of R&D -1.801
(FY 1995) (***)(.335)
University contractor 1.023
(***)(.228)
Lab fixed effects Excluded Included
significant
Number of observations 238 238
[R.sup.2] .495 .687
Dependent Variable
Estimated Fixed
Lab Effect from Total Citations
Column (2)(b) per Patent(c)
Independent Variables (3) (4)
Year = 1987 or later .783
(**)(.373)
Logarithm of R&D -.175
in fiscal year (.254)
Logarithm of patents(a) -.568
(**)(.278)
"Competition"(d) 1.687
(**)(.750)
"Focus"(e) 1.696 -1.045
(1.828) (2.443)
Change in "focus"(e) 1.116 2.815
(.834) (**)(1.311)
National security share of R&D -.477 -.840
(FY 1995) (.734) (.819)
Basic science share of R&D -1.722 -2.326
(FY 1995) (**)(.744) (**)(1.129)
University contractor .795 1.119
(.687) (*)(.645)
Lab fixed effects NA Excluded
Number of observations 17 205
[R.sup.2] .831 .096
Dependent Variable
Non-Self-Citations
per Patent(c)
Independent Variables (5)
Year = 1987 or later .444
(.330)
Logarithm of R&D -.130
in fiscal year (.229)
Logarithm of patents(a) -.524
(**)(.241)
"Competition"(d) 1.437
(**)(.663)
"Focus"(e) -1.555
(2.259)
Change in "focus"(e) 2.282
(**)(1.087)
National security share of R&D -.913
(FY 1995) (.784)
Basic science share of R&D -2.182
(FY 1995) (**)(1.019)
University contractor .952
(.578)
Lab fixed effects Excluded
Number of observations 205
[R.sup.2] .096
Dependent Variable
Estimated Fixed
Non-Self-Citations Lab Effect from
per Patent(c) Column (6)(b)
Independent Variables (6) (7)
Year = 1987 or later .666
(*)(.341)
Logarithm of R&D -1.696
in fiscal year (**)(.772)
Logarithm of patents(a) -.900
(***)(.312)
"Competition"(d) .970
(.743)
"Focus"(e) -2.801
(2.543)
Change in "focus"(e) .916
(1.134)
National security share of R&D -.228
(FY 1995) (.987)
Basic science share of R&D -1.173
(FY 1995) (.937)
University contractor 1.025
(.584)
Lab fixed effects Included NA
significant
Number of observations 205 17
[R.sup.2] .170 .969
(a) Logarithm of (DOE FFRDC patents + 1)/(weighted average
number of all patents by field) (see text for more detail).
(b) Projection of estimated fixed effect on lab characteristics
and means for time-varying regressors (see text).
(c) Normalized for truncation and variation in propensity
to cite as described in the text.
(d) Dummy equal to unity from 2 years before through 2 years
after change in contractor.
(e) Herfindahl Index of concentration of patents across
technology classes. The change is the difference between
the second and first half of the sample period.
Heteroskedasticity-consistent standard errors in parentheses;
(*), (**), (***) convey statistical significance at the 90%,
95% and 99% levels, respectively.
The time-invariant regressors drop out of the fixed-effects regression. Indeed, in the absence of additional data and identifying assumptions, their causal impact on the dependent variable is inextricably mixed up with that of the unobserved fixed effect. We can, however, ask to what extent the overall "lab effect"--which combines the effect of the observable characteristics and that of unobservable characteristics in an unknown way--is associated with the observable characteristics. We do this by first estimating the regression with dummy variables for each lab, and then projecting the estimated dummy coefficients on the observable lab characteristics and the mean for each lab of the time-varying regressors. The results of this projection tell us the "effects" of the characteristics on the dependent variables, estimated in a way that takes into account an unbiased estimate of the effect of the time-varying regressors, though we cannot really say whether this effect is causal or due to a common correlation with the unobserved lab characteristics.(25)
Looking first at patenting, column 1 shows the results of simple OLS on the pooled panel data, and column 2 shows the results of the fixed-effects regression. The dramatic increase in patenting associated with the policy shifts of the late 1980s is apparent in the large and statistically significant coefficient on the period commencing in 1987. All else equal, patenting after 1986 is approximately 50% greater than patenting before 1987, in both the OLS and fixed-effects versions. The results also show that patenting is related to R&D in the panel as a whole, although this relationship is not present in the "within" or time-series dimension. This is not surprising, given that we know overall patenting has been rising while R&D has been falling. The estimated effect of the competition variable on the rate of patenting is small and statistically insignificant.
Turning to the lab characteristics that do not vary over time, the OLS results in column 1 suggest that facilities with a greater basic science share have less patenting; the national security share is also negative but not statistically significant. More focused laboratories get more patents, all else equal, and those that decreased their focus the most have fewer patents.(26) These effects are quantitatively as well as statistically significant. Consistent with the benefits that Livermore derived from its association with the University of California's technology-transfer offices, facilities run by universities have almost twice as many patents, ceteris paribus, an effect that is statistically significant.
As noted, these estimates for the time-invariant characteristics are potentially biased, and we cannot control for unobserved lab characteristics as column 2 does for the time-varying regressors. The best we can do is, in effect, to purge them of bias due to the mutual correlation among the time-invariant characteristics, the time-varying regressors, and the fixed effect. This is done via a cross-sectional regression of the estimated fixed lab effects on the characteristics and the lab means for the time-varying regressors. These results are shown in column 3. They are qualitatively consistent with the OLS results in column 1, but the standard errors are much larger, and hence only the basic science share effect is statistically significant. Thus, overall the results for these characteristics are consistent with our conjectures, but we cannot confirm the direction of causality from these data.
For citations, columns 4 and 5 present simple OLS results for all citations, and also for data limited to non-self-citations. Column 6 presents the fixed-effects results (for the non-self-citations), and column 7 shows the projection of the fixed effects from this regression on the lab characteristics. The regressions have low explanatory power. Even with laboratory dummies, the [R.sup.2] is only .17. The dummy variable indicating that the observation is from 1987 or thereafter is positive but significant at the 95% level only for citations inclusive of self-citations. This is consistent with Figure 4, which showed a slight but uneven upward trend in normalized citations.
The R&D and patent variables have an interesting pattern of effect on citations. The patent variable has a significantly negative coefficient in both the OLS and fixed-effect regressions, providing some evidence that the "lower threshold" for patentability over time decreased the quality of patent applications. At the same time, R&D, which is essentially uncorrelated with citations (after controlling for patents) in the OLS, also has a significant negative coefficient in the fixed-effects regression. Taken together with the result for the patent variable, this says that years with above-mean patents have fewer citations, and years with above-mean R&D relative to patents also have fewer citations. Recall from Figure 2 that R&D was declining on average in the later years during which patenting was rising. Thus it is not clear whether the R&D and patent fixed-effect coefficients are telling us something "real" about the relationships among R&D, patent intensity, and citation intensity, or whether they are simply capturing a time pattern of policy changes that is more complex than our simple 1987-or-after dummy variable.
The competition variable has a positive and significant impact in the first two regressions. The point estimate of the magnitude is large, implying that patents applied for during the period of competitive pressures get an additional citation or so relative to what would otherwise be expected. This effect becomes smaller and statistically insignificant in the fixed-effects regression, which is not surprising given that the variation of the competition variable is mostly across laboratories rather than across time.
In the citation equations, the effect of "focus" is statistically insignificant. There is, however, a strong positive effective of the change in focus, at least in the simple OLS regressions in columns 4 and 5. That is, those laboratories whose focus decreased substantially saw a significant decline in their (normalized) citation intensity. We also find a statistically significant negative effect of basic science share, again in the OLS versions. None of these time-invariant characteristics have statistically significant impacts in the projection of the fixed effects, and, although none of them changes signs, some have significantly reduced magnitude. This makes it difficult to determine to what extent these factors are driven by correlations with R&D and patenting as distinct from a direct effect on citations.
Overall, the statistical results paint an interesting if slightly ambiguous picture of the technology-transfer process. There was a statistically significant increase in patenting, with no overall decrease in citation intensity. Within each lab there was an association between higher patenting and lower citations, and also between lower R&D and more citations. It is unclear whether these correlations are real or spurious. In the OLS regression there is a large, significant effect of university association on patenting, and a similarly large negative effect of diversification on both patenting and citation intensity; we cannot test for these effects in a way that controls for the presence of fixed effects. Finally, we find a suggestion in the panel of a positive effect associated with contractor turnover. It is unclear to what extent this is a selection effect, an incentive effect, or simply a "Hawthorne effect" in which management interest leads to an improvement in productivity even though nothing fundamental has changed.
[] Corroboratory analyses. One concern with the analyses above is that the patenting patterns may not reflect those in technology commercialization more generally. In particular, the cross-sectional patterns in patenting and citations to these patents may potentially not provide a complete depiction of the relative growth of technology-transfer activities at these facilities.
To address the concern, we undertook a similar regression analysis, using another measure of technology commercialization: the number of new CRADAs approved annually at each laboratory. We used observations between fiscal years 1991 (the first year with a significant number of CRADAs at DOE) and 1997 (with the exception of 1996, for which we had no data). Another possibility would have been to examine the dollar volume of corporate spending associated with these CRADAs. We rejected this approach for two reasons. First, the data on these expenditures were less complete. Second, laboratories and DOE subagencies differ dramatically in how the total expenditures associated with these transactions are compiled. In particular, while in some cases the corporation's in-kind contributions of time and facilities are included, in other cases only the direct financial payments to the laboratory are compiled.
To normalize the data, we employed in all regressions a dummy variable for each fiscal year. In this way, we hoped to control for DOE senior management's shifting emphasis on the importance of CRADAs (as discussed above, this was a major policy focus until the shift in congressional control to the Republican Party). Otherwise, the independent variables were the same as those in the patent equation. Because of the relatively small number of new CRADAs at each facility in a given year, we also tested whether the results were similar in a maximum-likelihood Poisson model, which explicitly treats the dependent variable as a nonnegative integer.
The results of the CRADA regressions, presented in Table 6, are largely consistent with the patent analyses. Not surprisingly, larger labs (as measured by R&D expenditure) have more CRADAs (columns 1 and 4). When controlling for fixed effects, the OLS finds no significant effect of R&D (column 2), but the Poisson formulation does find a significant R&D effect even with fixed effects (column 5). Labs that changed contractors ("competition") are labs with more CRADAs (column 1), but there is no significant connection within labs over time between CRADA activity and when the laboratory turnover occurred (columns 2 and 5).
TABLE 6 Panel Regression Results--CRADA Formation
Specification and
Dependent Variable
OLS OLS
Logarithm of Logarithm of
New CRADAs New CRADAs
Formed(a) Formed(a)
Independent Variable (1) (2)
Logarithm of R&D .897 -.009
in fiscal year (***)(.123) (.151)
"Competition"(c) .649 -.058
(**)(.320) (.267)
"Focus"(d) -1.822
(*)(.996)
Change in "focus"(d) -.855
(**)(.399)
National security -1.430
share of R&D (FY 1995) (***)(.463)
Basic science share -1.680
of R&D (FY 1995) (**)(.700)
University contractor .694
(.453)
Lab fixed effects Excluded Included
significant
Number of observations 95 104
[R.sup.2] or pseudo .744 .872
[R.sup.2] (for Poisson)
Specification and Dependent Variable
OLS Poisson Poisson
Estimated (ML) (ML)
Fixed Lab New New
Effect from CRADAs CRADAs
Column (2)(b) Formed Formed
Independent Variable (3) (4) (5)
Logarithm of R&D 1.045 .768
in fiscal year (***)(.216) (**)(.373)
"Competition"(c) .401 -.232
(.340) (.213)
"Focus"(d) -.986 -7.492
(1.344) (**)(3.765)
Change in "focus"(d) -1.081 .295
(**)(.500) (.594)
National security -2.220 -1.495
share of R&D (FY 1995) (***)(.339) (***)(.428)
Basic science share -2.499 -.805
of R&D (FY 1995) (**)(.829) (.624)
University contractor 1.352 .227
(***)(.408) (.377)
Lab fixed effects NA Excluded Included
significant
Number of observations 19 95 104
[R.sup.2] or pseudo .876 .736 .823
[R.sup.2] (for Poisson)
(a) Logarithm of (New CRADAs Formed + 1) (see text for more detail).
(b) Projection of estimated fixed effect on lab characteristics
and lab means for time-varying regressors (see text).
(c) Dummy equal to unity from 2 years before through 2 years after
change in contractor.
(d) Herfindahl Index of concentration of patents
across technology classes. The change is the difference
between the second and first half of the sample period.
Heteroskedasticity-consistent standard errors in parentheses;
(*), (**), (***) convey statistical significance at
the 90%, 95% and 99% levels, respectively.
All columns also include year dummies.
The diversification ("change in focus") variable is significant in the OLS regressions but not the Poisson one; the coefficient on focus is negative (in contrast to its positive coefficient in the patent equation) and statistically significant in the Poisson formulation. Both basic-science and national-security orientations have clearly negative associations with CRADA activity. Finally, the OLS indicates a positive association between CRADAs and having a university as lab contractor (statistically significant after removing the effect of the correlation between the fixed effect and R&D), but this is not significant in the Poisson formulation.
In addition to the patent and CRADA data, we obtained data on new licenses granted and total license revenues for 13 of the 23 laboratories in 1997. Not surprisingly, the largest laboratories dominate these absolute measures: Lawrence Livermore, Brookhaven, Oak Ridge, and Sandia. When we normalize licenses and license revenue per dollar of R&D, the standouts are much less clear. For instance, if we use the criterion of above-average performance in both the licensing and patenting dimensions, the winners are Ames, Livermore, Idaho, Oak Ridge, and Savannah River. Los Alamos and the Stanford Linear Accelerator are relatively poor performers. (The data indicate essentially no licensing activity for two of the laboratories, Fermi and Princeton.)
Despite the small number of observations, we ran a few diagnostic regressions with these data. Each of the licensing-based indicators was regressed on the change in the technological focus measure, a competition measure, the national security share, and a dummy for being in a modestly large metropolitan area.(27) In the unreported results, the coefficient on the competition variable is positive but never statistically significant. The change in the focus variable has an effect that is positive for all indicators but is at best marginally statistically significant. The national security share is negative in all regressions but not statistically significant. Location in a metropolitan area has no significant effect. These results are at least broadly consistent with the patent- and CRADA-based measures, and they address some of the concerns about the generality of these measures.
5. Conclusions
* This article has examined the commercialization of publicly funded research that is pursued in a little-studied but important environment, the national laboratory. The empirical and case study analyses suggest that the policy reforms of the 1980s had a dramatic and positive effect on technology commercialization. Patenting increased sharply. Although within labs there is some evidence of an association between greater patenting and a decline in patent quality, the overall increase in patenting does not seem to have been associated with an overall decline in quality, as was the case for universities.
Case studies suggest that the degree of technology-transfer success depends on bureaucratic factors, with beneficial effects of a university licensing office connection in one lab and a change in lab contractor in the other. The positive impact of a university connection is confirmed by the statistical analysis. There is some statistical evidence of an effect due to contractor turnover, but it cannot be pinned down precisely or modelled explicitly given the relatively few observations of contractor change.
These findings challenge the general picture of failure painted by earlier assessments of technology transfer at the national laboratories. The striking improvement in the measures of commercial activities at the laboratories, especially when compared to the experience of the universities, stands in contrast to the negative tone in many discussions such as the "Galvin Report" (Task Force, 1995). The apparent importance of limiting the distortionary effects of political interference has also not been heavily emphasized in many of the government studies. We do seem to confirm the adverse effect of diluting a laboratory's focus through extensive technological diversification.
We believe that these findings represent a first step toward a better understanding of the federal technology-transfer process. At the same time, it is important to acknowledge this article's limitations, which in turn suggest a variety of avenues for future research. First, the act of technology transfer from the national laboratory only begins the process of incorporating this technology into commercial innovation. To understand the impact on innovation and move toward estimates of rates of return, we need to look at the receipt or pickup of transferred technology in the private sector. While such an analysis presents conceptual and data challenges, the survey responses collected and analyzed by Adams, Chiang, and Jensen (1999) represent a promising first step in this direction. Ideally, we would like to know the private rates of return earned by companies seeking to commercialize national laboratory technology, and to compare these private rates of return with the social returns.
Second, the relative social costs and benefits from efforts to encourage technology-transfer activities remain unclear. National laboratories play a number of roles, of which producing technology for the commercial sector is only one. To what extent do efforts that encourage the licensing of new technologies and the spin-out of entrepreneurial businesses affect other laboratory activities negatively, by distracting researchers from their key missions? To what extent does the increase in commercial interactions actually inform and enhance more traditional R&D efforts? To answer these questions, it would be helpful to analyze the evolution of a broader range of measures of laboratory activities over time and across facilities. We found it surprisingly difficult to get data of this sort, and it seems that many performance measures are simply not tabulated systematically across laboratories. These questions are likely to be answerable only through in-depth case studies such as those of Mansfield et al. (1977).
Third, it would have been helpful to examine the incentives of laboratory researchers in more detail. An extensive literature (reviewed in Stephan (1996)) has documented how "career concerns"--e.g., monetary and nonmonetary awards--affect the behavior of researchers in universities. We know far less about the types of formal and informal incentive schemes present within government-owned laboratories. To what extent do formal reward schemes shape behavior? Are employment opportunities outside the laboratory important motivators? These questions are likely to be answerable only though detailed field research at particular laboratories.
Finally, we have analyzed only the contractor-operated laboratories at one agency, the U.S. Department of Energy. One of the consequential econometric issues that we faced in the article was the limited cross-sectional variation due to the modest number of laboratories and the lack of variability within our panel along possibly important dimensions. Comparing the experiences of facilities operated by different agencies, employing different organizational structures, and in different countries would all be logical extensions of this work. Such studies would also allow us to compare the relative effectiveness of a wide variety of initiatives to encourage technology transfer.
We thank Alper Afya and especially Petia Topaloya for excellent research assistance. Helpful comments were provided by Linda Cohen, Rosemarie Ham Ziedonis, Ariel Pakes (Editor), Sam Petuchowski, participants in the 1999 NBER Conference on Patent Policy and Innovation and the 2000 American Economic Association annual meeting, seminar participants at the Federal Reserve Bank of Boston and New York University, and an anonymous referee. Numerous officials at the DOE and the laboratories assisted in accessing data and contributed helpful insights, but especial thanks should go to Ray Barnes, Victor Chavez, Paul Gottlieb, Toni Joseph, Karena MacKinley, Gib Marguth, Ronald Meeks, Jeff Mobley, Bill Shepard, Warren Siemens, Claire Sink, and Walter Warnick. Thanks should go as well to Buddy Beck, Clyde Frank, and Thomas Widmer. We appreciate Linda Cohen's help in accessing CRADA data. Funding was provided by the NBER Project on Industrial Technology and Productivity with support from the Alfred P. Sloan Foundation and by Harvard Business School's Division of Research. All errors are our own.
(1) Examples include Nelson (1959), Jaffe (1989), and Henderson, Jaffe, and Trajtenberg (1998). See also Stephan (1996) and the references cited therein.
(2) Most of the academic literature has consisted of case studies of particular facilities (two thoughtful examples are Markusen and Oden (1996), and Ham and Mowery (1998)) and surveys of potential or actual users of laboratories (for example, Bozeman and Crow (1991), Roessner and Wise (1993), and Berman (1997)).
(3) One example is the 1998 controversy that stemmed from a reexamination proceeding by the U.S. Patent and Trademark Office concerning two patents covering low-power radar. One patent had been awarded to a small Alabama company, the other to Lawrence Livermore National Laboratory. Before the patent hearing was even held, Alabama Representative Robert "Bud" Cramer requested investigations of Livermore's technology-transfer activities by three congressional committees. Cramer also proposed legislation that would have restricted future collaborations between DOE FFRDCs and the private sector. Similar controversies have emerged over the years at a number of other laboratories.
(4) This issue was the topic of over forty congressional hearings and reports and four special commissions between 1940 and 1975 (U.S. Energy Research and Development Administration, 1976). Three historical overviews of the debates are Forman (1957), Neumeyer and Stedman (1971), and Hart (1998).
(5) This abbreviated history is based in part on Branscomb (1993), U.S. Office of Technology Assessment (1993), and U.S. General Accounting Office (1998). The interested reader is referred to these sources for more detailed accounts.
(6) These critiques date back at least as far as the "Bell Report" prepared by the Office of Management and Budget in 1963. Two clear expositions of these problems are found in the "Packard Report" (White House Science Council, 1983) and the "Galvin Report" (Task Force, 1995).
(7) It should be acknowledged that within even the earliest contracts granted by the Manhattan Engineer District during the war was a recognition that contractors should control nonatomic innovations that were only tangentially related to the government's mission. In practice, however, the government made very little use of this right to grant ownership to national laboratory contractors or to license its patents on an exclusive basis. (For a discussion, see U.S. Energy Research and Development Administration (1976).)
(8) For instance, U.S. Office of Technology Assess