Category: sources

Measuring Military Modernization

The U.S-China Security and Economic Review Commission (USCC) recently published a staff research paper entitled “Indigenous Weapons Development in China’s Military Modernization” that generated a great deal of media attention. One story noted that the report showed how the United States had “missed the emergence of significant military developments” and was “blinded by Beijing.” Another reportconcluded from the paper that “the United States has underestimated the growth of China’s military.”

What did the paper actually say?  It examined the development of four weapons systems: the Yuan-class (Type 041) diesel-powered submarine, the SC-19 anti-satellite missile, the DF-21D anti-ship ballistic missile, and the J-20 aircraft. Reviewing U.S. analysis of these development of these systems, the paper concluded that “there are no universal trends in publicly reported U.S. government analysis on the development of indigenous Chinese weapon systems.” According to the paper, only the emergence of the Yuan-class submarine was unexpected.  As for the other systems, the paper’s main contention is that analysts miscalculated the speed or rate of the development of these systems, but not their emergence.

The actual claims of the USCC paper are much more modest and mixed than media coverage would indicate. Most importantly, as the USCC notes but doesn’t emphasize, the degree to which the appearance of theYuan-class submarine in 2004 was a surprise remains contested. The USCC paper suggests that the U.S. missed this program because it wasn’t mentioned in the Pentagon’s 2003 annual report on Chinese military power. However, the 2002 version of this same Pentagon report clearly stated that “A new advanced version of the SONG-class conventional submarine is expected to incorporate advanced air-independent propulsion.” The Pentagon may not have provided the name of the class as we now know it, but it didn’t miss the development of a new submarine with the characteristics of the Yuan boats (which shares some design similarities with the Song).  Thus, the strongest example supporting the USCC paper’s criticism of analysis of the Chinese military in the United States is wanting.

As for assessments of the pace of weapons development, many U.S. government analysts and military officers have stated that development of the DF-21D was faster than expected.  Nevertheless, the system isn’t yet operational.  Adm. Willard stated in December 2010 that the development of the DF-21D had reached something equivalent to what the U.S. military defines as “initial operational capability.” The following month, however, another senior naval official noted that although the progress that had been achieved, the PLA wasn’t yet capable of “effectively employing the system” because it had not yet conducted a test over open water or been integrated into the force.

Other claims about such miscalculations are accurate, but perhaps not as consequential as the USCC paper and subsequent media coverage indicate. To start, the USCC paper acknowledges that the U.S. government accurately assessed the development of China’s anti-satellite missile. Instead, the paper’s claim is that analysts outside of government missed the mark. Still, the U.S. government didn’t overlook the emergence of this system or its development.

As for the J-20, the paper notes that progress has been slightly faster than originally estimated. In 2009, Defense Secretary Robert Gates remarked that China wouldn’t have a fifth-generation fighter until 2020. The following year, the U.S. revised its estimate for operational deployment to 2018.  The first prototype was launched a year earlier than the U.S. government expected in 2011, not 2012.  Nevertheless, the U.S. government has been tracking the development of the aircraft since 1997. Over the course of two decades of development, such miscalculations are not as great or dangerous as it might seem.

To be sure, China’s unwillingness to share information about its weapons programs is a major factor affecting assessments of their development. Such unwillingness is par for the course in any competitive relationship.

Yet, other reasons for the miscalculations cited by the paper are probably exaggerated.  The paper claims that analysts have underestimated “the extent of changes in the Chinese defense industry in the 1990s and early 2000s.” But back in 2005, analysts at RAND published a report using open sources that noted the progress that China had achieved in reforming its defense-industrial base precisely during this period.  Similarly, noted PLA expert Tai Ming Cheung documented these reforms in a 2009 book, Fortifying China. (Read this summary of Cheung’s findings.)  Neither publication, however, was cited by the USCC paper.

In addition, the paper claims that China’s threat perceptions have been systematically underestimated. In particular, the paper notes that analysts “may have failed to fully appreciate the extent to which the Chinese leadership views the United States as a fundamental threat to China’s security.” Yet the 2005 RAND report made exactly this point, noting in a subheading that “the PLA Leadership Perceives United States as Greatest Threat.” Drawing on open sources, Larry Wortzel (a USCC commissioner) came to a similar conclusion in a 2007 study: “China’s leaders and military thinkers see the United States as a major potential threat to the PLA and China’s interests.”

Finally, the severity of analytical failures can only be determined by understanding the degree of success. Apart from the anti-satellite missile, the paper doesn’t examine any indigenously systems whose development was accurately assessed. On this score, the U.S. government appears to have some important successes, especially regarding strategic weapons.

Take, for example, the DF-31 series of intercontinental ballistic missiles. The Pentagon noted in 2000 that “China…is designing a new generation of solid-fuel, road-mobile ICBMs,” and in 2002 that “deployment of the DF-31 should begin before mid-decade.” The Pentagon reported in 2006 that the missile had reached “initial threat availability” and in 2008 that it had been deployed to operational units.

Similarly, the Pentagon appears to have predicted quite accurately the development of China’s short and medium-range ballistic missiles. In a 1997 report on China’s military capabilities, the Pentagon estimated that China would have “the industrial capacity” to “as many as a thousand” such missiles. In its 2008 report to Congress, the Pentagon stated that the PLA had deployed “between 990 and 1,070” short-range ballistic missiles. In short, they nailed it.

The USCC staff research paper usefully draws attention to the challenges that analysts face when seeking track weapons development in China, a country that doesn’t seek to share such information. Further exploitation of Chinese-language open sources will certainly help improve future analysis. Nevertheless, the conclusions drawn in the media can’t be sustained by the paper, which itself deserves close scrutiny.

[This originally appeared on The Diplomat.]

Open Sources and Information Laundering

At the end of 2011, a study by the  at Georgetown University on China’s nuclear forces attracted a great deal of attention, including a  in The Washington Post.  The project documented the construction of networks of tunnels by the Second Artillery, the PLA’s strategic rocket forces, and suggested that China might have as many as 3,000 nuclear warheads – a figure roughly ten times higher than the current estimates of the U.S. government and independent research organizations such as the .

Experts and analysts quickly challenged many of the study’s most provocative claims about  and .  Questions were also raised about  that formed the basis of the study’s conclusions. A key source for the study’s claim about the number of warheads was .  To track down the original source for this claim, Greg Kulacki from the Union of Concerned Scientists, traveled to several libraries, including one in Hong Kong.  He learned that  was an article by a junior American naval officer published in 1986 whose data was reprinted in a 1995 Chinese-language magazine published in Hong Kong,  and subsequently recycled through online discussion forums for the next decade or so.

The Georgetown team trumpeted their use of open or unclassified Chinese-language sources of information as a new resource for the study of China’s military.  As other scholars and I , a veritable explosion of open-source information from China about military affairs has occurred, much of it online.  These sources include the PLA’s official publications such as newspapers, journals, and books, as well as unofficial sources, including popular magazines, online discussion forums, and unofficial websites.

Still, as Kulacki’s leg-work demonstrates, the proliferation of such open sources is a mixed blessing for scholars and analysts of the PLA.  On the one hand, more information is now accessible, which should improve the quality of research.  On the other hand, more information than any one individual can digest is now available – and mostly only in Chinese.

The volume of Chinese-language information now available places a premium on verifying the accuracy and authoritativeness of data from unofficial sources, especially blogs and online discussion forums.  Otherwise, the Internet becomes a convenient tool, unwittingly or not, for “information laundering” defined as concealing or masking the origins of a piece of data.

The seriousness of information laundering is evident throughout the Georgetown study.  Glancing through the slides of the presentation, one claim caught my eye: that the Second Artillery had 12 launch brigades facing India, including eight in the Tibetan Autonomous Region.  This was curious for two reasons.  First, the Second Artillery has only about thirty brigades, the majority of which are located in either China’s hinterland or coastal provinces, not near India.  Second, there are no confirmed reports of Second Artillery brigades inside Tibet.

The sources for this claim in the Georgetown study demonstrates how easily information can be laundered online.  The first source was a segment from a military news show from the Hong Kong-based Phoenix TV network. The second source was a post from a now defunct blog,  (that only appeared to be online for the month of November 2010 according to this ).  The blog post was redundant, as it only contained a crude machine translation of the transcript of the television segment.

The , however, was based not on information from China, but instead from India.  The segment was entitled “The Indian media states that the PLA has deployed 12 missile brigades on the border aimed at India” (??????????12????????). The source for the segment was reportedly an essay in the Times of India by a former Chief of the Army Staff of the Indian Army.   Yet a search of the Times of India’s website as well as the news database Lexis-Nexis couldn’t locate this essay.  As a result, the claim in the Georgetown study about Chinese missile brigades in Tibet was thoroughly laundered.  The original source for the claim can’t even be identified, much less verified.

Open-source information holds great promise for the study of China’s rapidly changing military.  But they must be used with great care, especially if the data comes from unofficial outlets such as blog posts and online forums.  Just because a piece of information about the PLA is available on the Internet in Chinese doesn’t endow it with authoritativeness in the absence of verification and corroboration.

UPDATE (10 January 2012): Through excellent sleuthing on the web, Time magazine’s Beijing correspondent has found what appears to be the Indian source of the Phoenix television segment.  It comes from , who served as Chief of the Army Staff in the late 1990s.  The op-ed, however, appeared in the Asian Age, not the Times of Indiaas Phoenix television reported.  It provides no source for the claim that the Second Artillery has launch brigades in Tibet.

[This post originally appeared on  as “”]

[This post was reprinted on as “”]