ANZAC Class Frigates — Sustainment

The audit objective was to examine whether the Department of Defence has effective and efficient sustainment arrangements for the Royal Australian Navy’s fleet of eight ANZAC class frigates.

Summary and recommendations

Background

1. The Royal Australian Navy (Navy) operates eight ANZAC class frigates. The frigates were commissioned between 1996 and 2006, and form part of Navy’s core surface warship capability. The ANZAC class is used to: conduct surveillance and patrols; protect shipping and strategic areas; provide naval gunfire in support of the Army; and undertake disaster relief and search and rescue activities.

2. The ANZAC class is half way through its original service life-of-type. The first frigate was expected to be withdrawn from service during 2024–25 and the last during 2032–33. In June 2018, the Australian Government announced that Hunter class frigates (under the SEA 5000 program) would replace the ANZAC class of ships, with the first Hunter class frigate scheduled to enter service in the late 2020s.1 To accommodate the design, build and introduction into service of the Hunter class frigates, the ANZAC class’ original withdrawal dates have been extended, with the first frigate to now be withdrawn in 2029–30 and the last in 2042–43.

3. The Department of Defence’s (Defence) Capability Acquisition and Sustainment Group is responsible for the sustainment of the ANZAC class. Navy has advised the Capability Acquisition and Sustainment Group of its requirements and budget for the sustainment of the ANZAC frigates in a Materiel Sustainment Agreement. The budget for the sustainment of the eight ANZAC class frigates for 2018–19 is $374.0 million — 15 per cent of Navy’s overall sustainment budget of $2,422.4 million for that year. The approved budget to sustain the ANZAC class from 2018–19 to 2026–27 is $3.4 billion.

Rationale for undertaking the audit

4. Defence’s sustainment of the ANZAC class frigates was selected for audit due to its cost and the importance of this capability until the Hunter class frigates enter into service. In addition, parliamentary committees have, over several years, stated their interest in Defence’s reporting of its sustainment performance and, in particular, obtaining greater insight into that performance.2

5. This audit is the fourth in a series of performance audits of Defence’s management of materiel sustainment:

  • Auditor-General Report No.44 2017–18 Defence’s Management of Sustainment Products — Health Materiel and Combat Rations;
  • Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment; and
  • Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements.

Audit objective and criteria

6. The audit objective was to examine whether the Department of Defence has effective and efficient sustainment arrangements for the Royal Australian Navy’s fleet of eight ANZAC class frigates.

7. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

  • Defence has a fit-for-purpose sustainment framework between Navy and the Capability Acquisition and Sustainment Group.
  • Defence has an appropriate framework to monitor and report on the effectiveness and efficiency of operating the ANZAC fleet.
  • Defence effectively administers the ANZAC sustainment strategic partnership to achieve specified availability and performance outcomes.

Conclusion

8. While the ANZAC class frigates are meeting Navy’s current capability requirements and continue to be deployed on operations in Australian, Middle Eastern and Asia-Pacific waters, Defence has been aware since at least 2012 that sustainment arrangements have not kept pace with higher than expected operational usage. Further, Defence cannot demonstrate the efficiency or outcomes of its sustainment arrangements, as the necessary performance information has not been captured. Defence will need to address relevant shortcomings in its sustainment arrangements to meet the requirement that the ANZAC class remain in service for an extra 10 years to 2043, pending the entry into service of the replacement Hunter class.

9. The effectiveness of Defence’s framework for sustaining the ANZAC class frigates has been reduced because the sustainment plans and budget outlined in the ANZAC class Product Delivery Schedule in Navy’s Materiel Sustainment Agreement do not align with the frigates’ higher than expected operational use. Defence has been aware of this misalignment since at least 2012.

10. Defence’s advice to the government to extend the ANZAC class’ life-of-type to 2043 was not based on a transition plan or informed by an analysis of the frigates’ physical capacity to deliver the required capability until then. Navy will need to address potential risks, relating to the frigates’ material condition, to maintain seaworthiness and capability.

11. Defence has established a performance framework for the ANZAC class frigates’ sustainment, with performance measures included in the Materiel Sustainment Agreement and reports provided to senior Defence leaders. While the performance measures adopted by Defence are relevant, the performance framework is not fully effective because the performance measures are:

  • only partially reliable — as targets and/or plans regularly change; and
  • not complete — as the measures do not address sustainment outcomes and efficiency.

In 2017–18 most of the Key Performance Indicators reported against were consistently not met.

12. The transparency of external reporting on the ANZAC frigates’ sustainment expenditure is reduced as it does not include Defence staffing costs or operational sustainment expenditure.

13. Defence entered into a sole sourced alliance contract with its existing industry partners, without a competitive process.

14. It is too early to assess the effectiveness of Defence’s administration of the new contracting arrangements, known as the Warship Asset Management Agreement, which took full effect in January 2018 after an 18-month transition period. Defence’s regular internal performance reporting and monitoring does not capture the performance of the Agreement.

Supporting findings

Sustainment framework

15. The ANZAC class Product Delivery Schedule in Navy’s Materiel Sustainment Agreement established with the Capability Acquisition and Sustainment Group is not fit-for-purpose. Navy has not updated the document to reflect the current governance arrangements and sustainment needs. The current sustainment plan and available budget do not accurately reflect the operational use of the frigates, which is higher than planned.

16. The misalignment between operational use and sustainment funding, combined with difficulties in securing necessary parts (in part, a result of obsolescence), has caused Defence to defer maintenance activities and transfer items of equipment between frigates.

17. Defence has identified the effects of the current misalignment between sustainment planning, funding and actual operational use. The ANZAC class has experienced degradation of the ships’ hulls and sub-systems, with successive reviews and performance information highlighting the link between lack of conformance to operating intent/requirement, reduced platform life and reduced sustainment efficiency.

18. In June 2018, Defence advised the Government of its intention to extend the planned withdrawal from service of the ANZAC class to 2043, indicating that a transition plan was due for completion in late 2019. The advice did not address the misalignment or assess the ANZAC class’ physical capacity to deliver the required capability until 2043. Defence is preparing a transition plan, which is due to be completed in late 2019, to guide the transition from the ANZAC class to the replacement Hunter class.

Performance monitoring and reporting

19. The performance measures adopted for the sustainment of the ANZAC class frigates are relevant but only partly reliable, as targets and/or plans regularly change. Further, the performance measures are not complete, as they do not address sustainment outcomes or efficiency.

20. Defence has established arrangements to monitor and report on the sustainment of the ANZAC class frigates, with senior Defence leaders made aware of the sustainment risks and issues experienced by the ANZAC class. The performance reporting indicates that there was underperformance for most of the Key Performance Indicators for the sustainment of the ANZAC class frigates during 2017–18. External reporting on the ANZAC class frigates’ sustainment expenditure would be more transparent if it included Defence staffing costs and operational sustainment expenditure.

Administration of the sustainment strategic partnership

21. Defence entered into a sole sourced alliance contract (the Warship Asset Management Agreement) with its existing industry partners, under an exemption from the Commonwealth Procurement Rules.

22. In the absence of a competitive process, Defence determined that value-for-money had been achieved after considering cost, the expertise of the industry partners, and their previous experience in sustaining the ANZAC class.

23. It is too early to assess the effectiveness of the contracting arrangements for ANZAC class sustainment, which took full effect in January 2018 after an 18-month transition period. The strategic partnership arrangement is expected to: drive efficiency; transfer risk to industry; reduce Defence’s cost of ownership; simplify contract administration; and reduce contract disputes. However, the arrangements may reduce Defence’s leverage over industry participants.

24. Defence entered into the new sustainment contract without seeking endorsement from the Defence Investment Committee or the Minister for Finance, on the assumption that ANZAC class sustainment had been approved at the time of the ships’ acquisition in the 1980s or possibly when they were introduced into service in the 1990s. Defence should have sought advice from central agencies on the most appropriate handling of this matter, given the high value of this procurement and the uncertainty over past approvals.

25. Defence’s regular internal performance reporting and monitoring does not capture the performance of the Warship Asset Management Agreement. The current misalignment of performance measures in the Warship Asset Management Agreement with the framework set out in the ANZAC class Product Delivery Schedule of the Materiel Sustainment Agreement may result in a lack of clarity around the achievement of outcomes.

26. Defence’s initial assessment of the performance of the Warship Asset Management Agreement indicates that all measures had been met or exceeded as at late 2017. Defence plans to evaluate the value-for-money of its contracting arrangements in 2020.

Recommendations

Recommendation no. 1

Paragraph 2.25

Defence update the ANZAC class Product Delivery Schedule of the Navy Materiel Sustainment Agreement to align sustainment plans for the ANZAC class frigates with their operational use and material condition.

Department of Defence response: Agree.

Recommendation no. 2

Paragraph 2.48

In the context of developing its transition plan for the ANZAC class life-of-type extension, Defence review the capital and sustainment funding required to maintain the ANZAC class frigate capability until 2043, and advise the Government of the funding required to meet the Government’s capability requirements for the class or the capability trade-offs to be made.

Department of Defence response: Agree.

Recommendation no. 3

Paragraph 3.10

Defence review the key performance measures for the ANZAC class frigates’ sustainment to ensure they are reliable and complete.

Department of Defence response: Agree with qualification.

Recommendation no. 4

Paragraph 4.21

To align with the strategic planning approach outlined in the Defence Integrated Investment Program, Defence develop guidance in the Capability Life Cycle Manual on when a proposal to establish or amend a sustainment program should be provided to the Defence Investment Committee and the Minister for Finance for consideration.

Department of Defence response: Agree with qualifications.

Recommendation no. 5

Paragraph 4.33

Defence refine its performance reporting and management arrangements for the ANZAC class frigates by aligning Key Performance Indicators in the Warship Asset Management Agreement and those in the ANZAC class Product Delivery Schedule of the Navy Materiel Sustainment Agreement.

Department of Defence response: Agree.

Summary of the Department of Defence’s response

27. The proposed audit report was provided to the Department of Defence, which provided a summary response that is set out below. The letter of response is reproduced at Appendix 1.

Defence welcomes the ANAO Audit Report into the ANZAC Class Frigates – Sustainment and agrees with the recommendations. Recommendations three and four have been agreed with qualifications.3

Defence would like to highlight the reliable performance and operational effectiveness of the ANZAC Class Frigates, and their ability to consistently achieve whole of government requirements during the previous two decades. Throughout the life of the ANZAC Class Frigates, Defence has effectively managed upgrades and subsequent sustainment of these warships in order to achieve the strategic requirements that have evolved since the introduction of the capability.

Defence is confident the assurance provided through this Seaworthiness regime affirms the warships are operational, seaworthy and capable of performing all assigned tasks. Furthermore, Defence is continually assessing options to optimise sustainment funding for the ANZAC Class Frigates to ensure operational availability and effectiveness continues to be met.

The Warship Asset Management Agreement (WAMA) has seen the implementation of greater cost oversight and improved performance based measures that encourage collaborative behaviours and a solutions focus within the industry partners. In line with the First Principles Review, the WAMA seeks to support long term relationships with industry that will underpin the sovereign capabilities essential to deliver continuous shipbuilding and sustainment.

Defence is actively planning and making preparations for the transition from the ANZAC Class Frigates to the Hunter Class Frigates to ensure effective operational coverage in a complex and ever changing strategic environment.

Key messages from this audit for all Australian Government entities

28. Below is a summary of key messages, including instances of good practice, which have been identified in this audit that may be relevant for the operations of other Australian Government entities.

1. Background

Introduction

1.1 The Royal Australian Navy (Navy) consists of around 46 commissioned vessels and over 14,000 personnel. Navy’s core surface warship capability includes eight ANZAC class frigates (FFH — frigate helicopter) which were commissioned between 1996 and 2006.4 The frigates were based on the German Meko 200 design and were built by Tenix Defence (now BAE Systems) at the Williamstown shipyard in Melbourne.5

Figure 1.1: HMAS Stuart (FFH-153), Navy ANZAC class frigate

Source: Navy.

1.2 The ANZAC class is a long-range frigate and is used to: conduct surveillance and patrols; protect shipping and strategic areas; provide naval gunfire in support of the Army; and undertake disaster relief and search and rescue activities. Since their introduction into service, each frigate has undertaken multiple deployments, including to South East Asia, the Middle East and the Pacific. Appendix 2 outlines key deployments and dates for each of the eight frigates.

1.3 The ANZAC class is half way through its original service life-of-type. The first frigate was expected to be withdrawn from service during 2024–25, with the last frigate to be withdrawn during 2032–33. In June 2018, the Australian Government announced that the Hunter class frigates (under the SEA 5000 program) would replace the ANZAC class of ships, with the first Hunter class frigate to enter service in the late 2020s.6 To accommodate the design, build and introduction into service of the replacement Hunter class frigates, the ANZAC class’ original withdrawal dates have been extended, with the first frigate to now be withdrawn in 2029–30 and the last frigate in 2042–43.

Sustainment arrangements for the ANZAC class frigates

1.4 The ANZAC frigates are a ‘Top 30’ sustainment product for the Department of Defence (Defence).7 In 2017–18, Navy spent $341 million on the sustainment of the ANZAC class, second only to the annual sustainment costs of the Collins class submarines at $622 million. The budget for the sustainment of the eight ANZAC class frigates for 2018–19 is $374 million — 15 per cent of Navy’s overall sustainment budget of $2,422 million for that year.8

1.5 Defence’s Capability Acquisition and Sustainment Group (CASG) is responsible for the sustainment of the ANZAC class.9 Navy has advised CASG of its requirements and budget for the sustainment of the ANZAC class frigates in a Materiel Sustainment Agreement.10 The agreement also sets out the performance information Navy requires to obtain assurance that the ANZAC class frigates are being sustained to meet Navy’s planned operational use of the ships.11

1.6 Within CASG, the ANZAC Systems Program Office has been established to sustain the ANZAC class frigates, including the integration of any changes from Navy capital projects (such as system upgrades) into the ANZAC class frigates. Since the ANZAC class was introduced into service, the ANZAC Systems Program Office has largely outsourced the sustainment of the class to industry.12 In July 2016, Defence contracted BAE Systems Australia, Saab Australia and Naval Ship Management (Australia) to sustain the frigates through the ‘Warship Asset Management Agreement’. Figure 1.2 outlines Defence’s current sustainment arrangements for the ANZAC class frigates.

1.7 As well as routine maintenance activities during deployment, the ANZAC class frigates’ sustainment has included deep-cycle maintenance activities such as:

  • upgrading the Anti-Ship Missile Defence systems, including replacement of the mast and radar. The first ship was upgraded in 2010, with the trials completed by mid-2011. The upgrade of the remaining ships occurred between 2012 and 2017; and
  • the ANZAC Mid-life Capability Assurance Program. The program began in September 2017 and the final ship is scheduled to be upgraded by 2023. The upgrade is to address obsolescence and incorporate projects SEA 1408 Phase 2 Torpedo Self Defence, SEA1442 Phase 4 Maritime Communications Modernisation, and SEA 1448 Phase 4B Air Search Radar Replacement into the frigates. Figure 1.3 shows HMAS Arunta (FFH-151) undergoing maintenance under the ANZAC Mid-life Capability Assurance Program at the Henderson shipyard in July 2018. A hole has been cut in the port side of the hull to allow for the removal and re-installation of parts and sub-systems.

1.8 Scheduled deep-cycle maintenance programs contribute to the maintenance of and capability and provide an opportunity to assess the material state of the ANZAC class frigates’ hull and sub-systems.

Figure 1.2: Defence sustainment arrangements for the ANZAC class frigatesa

Note a: Several other areas within Defence also contribute to the delivery of ANZAC sustainment, including cross platform products. Additionally, the ANZAC Systems Program Office has several minor supplier contracts with industry.

Source: ANAO analysis of Defence documentation.

Figure 1.3: Access point on the port side of HMAS Arunta’s (FFH-151) hull during maintenance at Henderson shipyard in July 2018.

A photograph of ANZAC class frigate HMAS Arunta undergoing maintenance at the Henderson shipyard in Western Australia.

Source: ANAO site visit to Henderson shipyard — July 2018.

Rationale for undertaking the audit

1.9 Defence’s sustainment of the ANZAC class frigates was selected for audit due to its cost (as discussed in paragraph 1.4), and the importance of this capability until the Hunter class frigates enter into service. In addition, parliamentary committees have, over several years, stated their interest in Defence’s reporting of its sustainment performance and, in particular, obtaining greater insight into that performance.13

1.10 This audit is the fourth in a series of performance audits of Defence’s management of materiel sustainment:

  • Auditor-General Report No.44 2017–18 Defence’s Management of Sustainment Products—Health Materiel and Combat Rations;
  • Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment; and
  • Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements.

Audit approach

Audit objective, criteria and scope

1.11 The audit objective was to examine whether Defence has effective and efficient sustainment arrangements for the Royal Australian Navy’s fleet of eight ANZAC class frigates.

1.12 To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

  • Defence has a fit-for-purpose sustainment framework between Navy and the Capability Acquisition and Sustainment Group.
  • Defence has an appropriate framework to monitor and report on the effectiveness and efficiency of operating the ANZAC fleet.
  • Defence effectively administers the ANZAC sustainment strategic partnership to achieve specified availability and performance outcomes.

Audit methodology

1.13 This audit focussed on sustainment governance, contract management and performance management arrangements for the ANZAC class frigates. In undertaking the audit, the ANAO:

  • reviewed relevant Defence files and documentation;
  • collected and analysed data relating to the sustainment of the ANZAC class frigates; and
  • interviewed key personnel from Defence and industry, and visited sustainment operations in Western Australia in July 2018.

1.14 The audit examined the current Warship Asset Management Agreement (effective 1 July 2016). The audit did not examine the prior contracts for sustainment services.

1.15 As discussed in paragraph 1.7, the ANZAC class frigates were upgraded as part of the Anti-Ship Missile Defence Project, intended to improve the class’s anti-ship self-defence capability (known as SEA1448 Phase 2A and Phase 2B, with further phases scheduled). This project was not examined as part of the audit.14

1.16 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $451,000. The team members for this audit were Alex Wilkinson, Megan Beven, Zak Brighton-Knight and Sally Ramsey.

2. Sustainment framework

Areas examined

This chapter examines whether the Department of Defence’s (Defence) framework for sustaining the ANZAC frigates is fit-for-purpose. It considers the Materiel Sustainment Agreement between the Royal Australian Navy (Navy) and the Capability Acquisition and Sustainment Group; and the alignment between sustainment planning, budget, the operational use of the ships, and their physical condition.

Conclusion

The effectiveness of Defence’s framework for sustaining the ANZAC class frigates has been reduced because the sustainment plans and budget outlined in the ANZAC class Product Delivery Schedule in Navy’s Materiel Sustainment Agreement do not align with the frigates’ higher than expected operational use. Defence has been aware of this misalignment since at least 2012.

Defence’s advice to the government to extend the ANZAC class’ life-of-type to 2043 was not based on transition plan or informed by an analysis of the frigates’ physical capacity to deliver the required capability until then. Navy will need to address potential risks, relating to the frigates’ material condition, to maintain seaworthiness and capability.

Areas for improvement

This chapter includes two recommendations intended to improve the sustainment framework by: aligning sustainment plans with the operational use and material condition of the ANZAC class frigates; and reviewing the capital and sustainment funding required to extend the life of the ANZAC class frigate capability until the anticipated withdrawal of the final frigate in 2043.

In addition, consideration should be given to reviewing and updating the documentation underpinning the framework, in particular the Standard Procedure document for Materiel Sustainment Agreements and the Navy Heads of Agreement.

Has Navy established a fit-for-purpose Materiel Sustainment Agreement for managing the sustainment of the ANZAC class frigates?

The ANZAC class Product Delivery Schedule in Navy’s Materiel Sustainment Agreement established with the Capability Acquisition and Sustainment Group is not fit-for-purpose. Navy has not updated the document to reflect the current governance arrangements and sustainment needs. The current sustainment plan and available budget do not accurately reflect the operational use of the frigates, which is higher than planned.

The misalignment between operational use and sustainment funding, combined with difficulties in securing necessary parts (in part, a result of obsolescence), has caused Defence to defer maintenance activities and transfer items of equipment between frigates.

2.1 Since 2005, the Department of Defence (Defence) has used Materiel Sustainment Agreements to formalise the relationship between the Services (Australian Army, Royal Australian Navy and Royal Australian Air Force) and Defence’s Capability Acquisition and Sustainment Group (CASG).15 As of October 2018, Defence had eight active Materiel Sustainment Agreements. The Royal Australian Navy’s (Navy) agreement had 34 Product Delivery Schedules attached.16 One of these Product Delivery Schedules sets out Navy’s requirements for the sustainment of the ANZAC class frigates.

2.2 Defence’s overarching policy on Materiel Sustainment Agreements is set out in the Standard Procedure on Materiel Sustainment Agreements.17 The procedure provides direction on tasks and activities associated with the management of Materiel Sustainment Agreements. First introduced in 2012, the Standard Procedure document had an agreed review period of 12 months. However, as at October 2018, the procedure had not been reviewed. The procedure does not, for example, reflect the organisational governance changes made by Defence following the 2015 Creating One Defence First Principles Review. There would be merit in undertaking the planned review of the Standard Procedure document.

The Heads of Agreement

2.3 The sustainment business model outlined by Navy in its Materiel Sustainment Agreement Heads of Agreement seeks three outputs from CASG:

  • materiel availability: the provision of the product for Navy use, at its agreed specification and configuration baseline, and in accordance with seaworthiness and airworthiness requirements.18 Materiel availability addresses the immediate issue of preventative and corrective maintenance that delivers quality Materiel Ready Days19 or their equivalent;
  • materiel confidence: contribute to seaworthiness and airworthiness through actions to ensure that the products will remain available at their agreed performance specification through the life of the product; and
  • sustainment efficiency: use of the funds available (expenditure against plan to the allocated budget) and the actions to improve cost-effective use of those funds.

2.4 The Navy Heads of Agreement was updated annually until 2012. Since then it has been updated once, in early 2015, to ‘incorporate lessons learned from experience with post-Rizzo [Materiel Sustainment Arrangement] management arrangements.’20 Defence should consider updating the Navy Heads of Agreement to reflect the current departmental structure.

Product Delivery Schedule

2.5 The Product Delivery Schedule (which is attached to the Materiel Sustainment Agreement) identifies: the outputs expected from CASG in terms of the sustainment of the ANZAC class frigates; Key Performance Indicators and Key Health Indicators; risks that have the potential to limit the successful achievement of Navy’s three required outputs; and the key requirements and responsibilities of Navy and CASG. For example, CASG is required to deliver:

the cost effective, sustainable and safe:

  • maintenance of the product, including sub systems and support systems, through life to the configuration standard and system specifications;
  • provision of spares and support to support operations;
  • integration of new capabilities and implementation of obsolescence remediation as approved and funded; and
  • identification of obsolescence and initiatives to reduce costs over the life of the platform.21

2.6 Delivery of the ANZAC class frigates’ sustainment against the schedule is to be monitored and reported on monthly to an Operational Sustainment Management Meeting through Defence’s Sustainment Performance Management System. Performance measurement and reporting is discussed in Chapter 3 of this Report.

Funding for sustainment activities

2.7 Sustainment funding consists of baseline funding allocated from Defence’s Capability Sustainment Program and operational funding allocated for the additional cost of major Defence operations.22 The approved baseline funding in the ANZAC class Product Delivery Schedule does not include the cost of sustainment activities that Navy’s Fleet Support Unit is to deliver as part of the Warship Asset Management Agreement (the WAMA or the Alliance).23

2.8 The Product Delivery Schedule sets out the whole-of-life costing to 2027–28 for the sustainment of the ANZAC class as $3.61 billion. Approved baseline funding set out in the schedule comprises sustainment funding of $3.40 billion until 2027–28 with $374.37 million allocated for 2018–19. The schedule indicates a funding shortfall of $212.92 million for the period to 2027–28.24 Figure 2.1 outlines the estimated required baseline funding, Navy approved baseline funding and cumulative shortfall for ANZAC class sustainment.

Figure 2.1: Estimated required baseline funding, Navy approved baseline funding and cumulative shortfall for ANZAC class sustainment, as at July 2018a

Note a: The ANZAC Mid-life Capability Assurance Program concludes in 2022–23.

Source: ANAO analysis of Defence documentation.

2.9 Defence has identified a need to review these funding arrangements. In the March 2016 fleet screenings25for ANZAC class sustainment, the Capability Manager for the Surface Combatant group concluded that:

  • Without an increase to CN02 [ANZAC class sustainment] funding allocation, the ANZAC class capability will suffer a degradation in system and platform reliability and availability that will impact the lethality at the unit level and inhibit the ability of the FFH [ANZAC class frigate] to meet the requirements of the Navy Warfighting Strategy 2018 through to PWD [planned withdrawal date].
  • The significant funding shortfalls across the CN02 [ANZAC class sustainment] PDS [Product Delivery Schedule] across the DMFP [Defence Management and Financial Plan] for sustainment of the ANZAC Class at the current expected level of capability. This under-funding extends to PWD [planned withdrawal date].
  • The MEDIUM level of confidence in the sustainment and EC [engineering change] over the forward estimates.
  • There is a significant backlog of EC [engineering change] installations as a result of deferrals and prioritisation which now require execution due to sustainment or obsolescence issues reaching critical thresholds. The majority of the ECP [engineering change program] is within the unfunded component of the CN02 [ANZAC class sustainment] budget.

2.10 Subsequent fleet screenings in November 2017 and March 2018 reiterated these conclusions. In July 2018, the life-of-type for the ANZAC class was extended to 2043, over 10 years longer than the original planned withdrawal date (see Figure 2.2 at paragraph 2.41). The planned withdrawal date referred to in the 2016 and 2018 fleet screenings was prior to the extension of life-of-type of the ANZAC class to accommodate the development and build of the replacement Hunter class frigates. In light of the extension, there would be merit in reviewing the ANZAC class sustainment funding estimates. The 2011 Rizzo review, for example, highlighted the need for Defence to consider the increased maintenance costs of a platform towards the end of its life-of-type.26

Operating requirements

2.11 The Product Delivery Schedule records Navy’s materiel requirements including the Product Operating Profile and the Product Activity Plan. The Product Operating Profile describes the employment profiles and Navy’s planned rate of usage, providing the context in which the frigates are used. The Product Activity Plan provides a schedule of monthly Materiel Ready Days27 for each ship for the duration of the Product Schedule, and the maintenance periods necessary to deliver the requirements of the Product Delivery Schedule.28

2.12 The Operating Profile informs CASG’s sustainment planning and the development of costings against the Product Operating Profile. Since June 2012, each financial year’s Product Delivery Schedule for the ANZAC class frigates (the latest update being issued in July 2018) has stated that the Product Operating Profile:

has deviated from the initial functional specification described during the ANZAC Ship Project acquisition phase. As a result the current sustainment plans and budget for the Product does not accurately reflect the current state of operational use.

2.13 The misalignment between operational use and sustainment planning and funding, combined with difficulties in securing necessary parts (in part, a result of obsolescence) has caused Defence to defer maintenance activities and transfer items of equipment between frigates.

Deferral of External Maintenance Period tasks

2.14 In 2015, Navy delayed expenditure on three ANZAC class frigates’ sustainment to address funding pressures. HMAS ANZAC (FFH-150) had $600,000 of maintenance tasks deferred, and HMAS Perth (FFH-157) and HMAS Ballarat (FFH-155) each had $3 million of maintenance tasks deferred. Navy was aware that delaying expenditure on maintenance tasks would ‘result in deferred maintenance creating a large body of outstanding work and associated cost and risk to seaworthiness’.29

2.15 In July 2017, Defence internal reporting identified that:

Maintenance deferrals have at times been conducted without appropriate assessment and approvals. Individually these examples are being addressed; however the sporadic non-compliance with Alliance processes and Defence Policy is considered a risk to seaworthiness for the Class.

2.16 In November 2018, Defence advised the ANAO that ‘all deferred maintenance is subject to individual risk assessments as documented within the ships’ respective Moratorium Letter’. Defence further advised in February 2019 that:

Ship maintenance performance is continuously reviewed jointly by CASG and Navy. Whole of fleet risk is continually assessed, managed and assured under the Navy Seaworthiness regime which considers and addresses individual Class Materiel Mission Risk assessments.

Transfer of items and equipment between frigates

2.17 In 2015, Navy identified that increased use of the frigates had depleted maintenance stocks and the transfer (cannibalisation) of parts from one frigate to service another had increased. In October 2016, the Deputy Chief of Navy was advised by Director General Major Surface Ships and Capability Manager Surface Combatant Group that:

To maintain the current required level of capability for the FFH [Anzac class frigates], it is necessary to transfer items and equipment between platforms. The main reasons for this requirement are: Item is obsolete and no longer available; or Item is not available off the shelf and Procurement lead-time does not meet operational requirement.

2.18 In August 2017, Defence internal analysis identified that for the financial year 2016–17 there had been:

… a decreasing trend for cannibalisations. This is not represented well by the rolling average performance target. A strong correlation exists between peaks in cannibalisation and ships leaving extended maintenance availabilities. The cessation of inventory transfers is resulting in a stabilisation of cannibalisation numbers around the lower level presented here for the past four months… Further improvement will result from continuation of the management plan developed in support of replacing all items cannibalised from HMAS Perth, and analysis to identify causal factors and prevent recurrences.

2.19 In November 2018, Defence advised the ANAO that:

During the Anti-Ship Missile Defence (ASMD) upgrade the number of cannibalisations was large, however it has decreased across the class since the completion of the upgrade program in mid 2017.

… the rate of Cannibalisations has decreased from a monthly average of 17.33 in October 2016 to a monthly average of 3…

2.20 Defence internal reporting in June 2017 identified that the spike in cannibalisations during the Anti-Ship Missile Upgrade program was caused by each successive ship entering the program being ‘used as the next ‘equipment donor’, which represented a risk of cannibalisations increasing towards completion of the program’.

2.21 Table 2.1 shows the reported instances of cannibalisations from 2015–16 to 2017–18.

Table 2.1: Reported instances of cannibalisations, 2015–16 to 2017–18a

Financial year

Number of cannibalisations

2015–16

123

2016–17

125

2017–18b

67

Note a: The ANAO notes that the last ships completed the Anti-Ship Missile Defence Project in 2017.

Note b: HMAS Perth (FFH-157) was in lay-up during 2017–18.

Source: ANAO analysis of monthly reported instances of Sustainment Performance Reporting System results for Key Health Indicator ‘cannibalisation’.

2.22 Box 1 discusses the reasons for cannibalisation in the Defence context and the findings from the United Kingdom National Audit Office’s audit of cannibalisation of naval platforms within the Royal Navy.

Box 1: Cannibalisation of naval platforms

The United Kingdom’s National Audit Office undertook an audit of cannibalisation of ships within the Royal Navy. The audit was produced ‘against a background of wider concerns about the affordability of the [Ministry of Defence’s] equipment and support plans, and consideration of the forthcoming changes to how the [Royal] Navy will operate as a myriad of new vessels are brought into service’.

The National Audit Office observed that cannibalisation can be an effective way to manage operational and maintenance priorities; however, it can also ‘lead to increased costs and disruption, divert resources from other activities and create additional technical and financial risks.’ The Royal Australian Navy’s Materiel Sustainment Agreement Performance Framework notes that ‘cannibalisation reflects an inability of the supply chain to support requirements’ though ‘is not an indicator of supply chain performance alone as the requirement may be the result of induced failure from a number of domains.’

Defence documentation indicates cannibalisation occurs, for example, when there is: a shortage of available parts or extensive lead times; uncodified items (and not supported by the supply chain); no serviceable stock, with some items unable to be repaired due to funding constraints or repaired by required delivery date; obsolescent parts; and an operational need.

The National Audit Office found that:

  • Cannibalisation can be necessary but should only happen when no other solution is available.
  • Each instance of cannibalisation can delay programmes, create additional engineering risks and add to the work of staff, affecting morale.
  • 71 per cent of parts cannibalised on the basis of operational need were low-value, but the cost of moving the parts could be much greater.
  • The need for cannibalisation is exacerbated by both a lack of information about when parts will be delivered, and delays in receiving parts on time.30

2.23 Navy’s experience from the ANZAC class Anti-Ship Missile Defence upgrade indicated that the cannibalisation of equipment costs up to three times more than undertaking a standard repair methodology such as sourcing a replacement from a supplier or repairing the part.

2.24 The effective sustainment of the ANZAC class frigates until 2043 requires alignment between sustainment plans outlined in the Product Delivery Schedule and the ships’ operational usage and material condition. Defence has been aware, since at least 2012, that the ANZAC class frigates’ sustainment arrangements are not aligned with the ships’ requirements. Defence should update the ANZAC class Product Delivery Schedule to help address this misalignment.

Recommendation no.1

2.25 Defence update the ANZAC class Product Delivery Schedule of the Navy Materiel Sustainment Agreement to align sustainment plans for the ANZAC class frigates with their operational use and material condition.

Department of Defence’s response: Agree.

Has Defence assessed the effect of the misalignment of the sustainment plans and budget with the ANZAC class frigates’ operational use?

Defence has identified the effects of the current misalignment between sustainment planning, funding and actual operational use. The ANZAC class has experienced degradation of the ships’ hulls and sub-systems, with successive reviews and performance information highlighting the link between lack of conformance to operating intent/requirement, reduced platform life and reduced sustainment efficiency.

In June 2018, Defence advised the Government of its intention to extend the planned withdrawal from service of the ANZAC class to 2043, indicating that a transition plan was due for completion in late 2019. The advice did not address the misalignment or assess the ANZAC class’ physical capacity to deliver the required capability until 2043. Defence is preparing a transition plan, which is due to be completed in late 2019, to guide the transition from the ANZAC class to the replacement Hunter class.

Assessment of condition

2.26 A 2015 Gate Review31 of the ANZAC sustainment operation identified a range of challenges for future support of the ANZACs, including the state of the ships and related assurance processes:

Future support to ANZAC faces a number of challenges, in particular apparently unconstrained budget growth. This budget growth is, at least in part, due to uncertainty regarding the actual materiel state of the ships, concerns with corporate supporting systems that may compromise the assurance of their technical integrity, ongoing upgrade programmes and planned changes to the current operating model.

2.27 Defence is currently determining the state of its ANZAC class frigates through a program of mid-life upgrades to the class. An internal Defence study into the ANZAC class in late 2017 observed that:

Unfortunately, accurate records have not been maintained as to the long-term usage of individual equipment items over the life of each ship nor the stresses to which the hull and other systems have been subjected.32

2.28 Defence records indicate that it was aware prior to the mid-life upgrade that the frigates’ hulls and sub-systems had degraded. For example:

The degraded material state of the vessels has been exposed through the ASMD [Anti-Ship Missile Defence] refit and upgrade. This is particularly evident in the poor condition of the hull and the high growth in corrective maintenance across platform systems and has resulted in a significant maintenance cost increase.33

2.29 Defence identified that the following ‘causal factors’ had contributed to the condition of the ANZAC class ships:

  • an inadequate initial sustainment model;
  • operating beyond the original Statement of Operating Intent;
  • changes to the expected capability;
  • additional cost of operating as a ‘parent navy’ for the ANZAC class;34 and
  • an increase in required sustainment.
An inadequate initial sustainment model

2.30 In 2015 Defence observed that:

At procurement, the ANZAC Class sustainment model delivered at IIS [introduction into service] has proven to be inadequate to fund the capability and therefore the FFHs [ANZAC class frigates] have been underfunded since IIS [introduction into service]. Navy is continuing to manage the latent symptoms of the behaviours around associated sustainment and capability decisions made over the last 20 years of the life of the ANZAC Class. These are the same behaviours which were highlighted in the Rizzo Review.

2.31 Box 2 summarises the key findings of the 2011 Rizzo Review into ship support and management practices.

Box 2: Summary of the findings made in the Rizzo Review regarding the Kanimbla class landing platform amphibious ships

HMAS Kanimbla (L-51) and HMAS Manoora (L-52) were Navy’s Kanimbla class landing platform amphibious ships. The class was subject to high operational tempo and deferred maintenance. These issues, as well as the age of the ships, contributed to the Kanimbla class: accumulating large quantities of corrosion; faults with the deck crane and alarm system; the need to overhaul propulsion machinery, power generators, and air conditioning; and an outdated communication suite.

These issues resulted in the Kanimbla class being unable to assist in the aftermath of Cyclone Yasi in 2011 and being decommissioned early due to it being not cost-effective to repair the ships. This diminished Navy’s amphibious and transport capabilities and led to the purchase of interim ships to cover the capability gap.

In response to the issues identified in the Kanimbla class ships’ sustainment, the Australian Government commissioned the Plan to Reform Support Ship and Management Practices, commonly referred to as the ‘Rizzo Review’, completed in July 2011. The review made 24 recommendations aimed at improving Navy’s sustainment operations and capability lifecycle management. Importantly the review found:

For many years, preventative and corrective maintenance has not been carried out because of the higher priority afforded to operational demands over maintenance requirements. The risks involved in deferring maintenance are not fully appreciated by non-engineering officers in Navy… Failure occurred as a result of Navy systems that are inadequate and under-resourced and cultural difficulties that compromised standards and placed an overwhelming focus on achieving the operational program.35

Operating beyond the original Statement of Operating Intent

2.32 The way Defence has operated the ANZAC class frigates since acquisition has increased the usage of systems and equipment beyond the original design intent, accelerating the ageing of the ships systems and increasing early obsolescence. Defence’s monthly internal reporting between 2015–16 and 2017–18 reported performance against the ‘conformance to operating intent’ Key Performance Indicator as ‘red’ for 26 of the 36 months due to operation of the platform outside:

  • the operating environment defined in the Statement of Operating intent (for example, outside the design specification for ambient sea water temperatures);
  • the operating profile (for example, operating speed) defined in the Statement of Operating intent; and/or
  • number of permitted sea days over a two year period.

2.33 The operation of the frigates was considered to be outside the operating intent or design due to:

  • a 20 per cent increase in crew size from 157 to 192 and an increased endurance from 30 to 36 days, which had increased the workload on systems including sewage treatment, water generation, refrigeration, power generation and air conditioning;
  • an increase in operational tempo from 125 to 150 days per annum, which had increased the running hours of systems;
  • variance in operation from the baseline design — the Meko 200 baseline design for the frigates was based on operations in a cool climate and deep water, whereas the ANZAC class frigates have operated for extended periods in warm areas in coastal and archipelagic regions (see Appendix 2); and
  • a 50 per cent increase in required power due to modifications made to the ship since introduction into service and major system upgrades.

2.34 Navy internal guidance on the development of performance measures in July 2016 stated that ‘there is a proven link between lack of conformance to operating intent/requirement and reduced platform life/reduced sustainment efficiency’.

Changes to the expected capability

2.35 In November 2015, the Deputy Chief of Navy was advised that:

…additional equipment fits [to the ANZAC class frigates] were not installed under a fully developed engineering change.

2.36 For example, installation of additional systems, such as the anti-ship missile defence system, have increased the frigates’ displacement from 3600 tonnes to 3900 tonnes. As a result, the frigates’ propulsion diesel engines have operated at full power for much longer than originally intended. These additions and operational changes have also increased the cost of ownership. In November 2018, Defence advised the ANAO that:

Additional equipment fitted to the ANZAC class frigates after their introduction into service has realised the designed capability and incorporated additional contemporary capability to ensure the lethality of the ANZAC Class. Having followed a fully developed engineering change process (for example, installation of additional systems, such as the anti-ship missile defence system) the through life margins of the platform have been consumed at a faster rate than anticipated when originally placed into service in 1996. This has resulted in a margin recovery program that has increased the frigates’ displacement from 3600 tonnes to 3900 tonnes. As a result, the frigates’ propulsion diesel engines are required to provide higher power and increased fuel consumption to achieve a given speed, and that the maximum speed has been marginally reduced. These additions and operational changes have consequently increased the cost of ownership.

Additional cost of operating as a ‘parent navy’ for the ANZAC class

2.37 The unique Australian modifications made to the frigates, in particular the addition of the Anti-Ship Missile Defence System and the Australian combat system, have distanced the class from its original design:

There are additional costs associated with Australia being a Parent Navy of the ANZAC Class, particularly associated with design and installation of engineering change (non-recurring engineering costs), either due to obsolescence or a change to the Functional Baseline. As a Parent Navy for the FFH [ANZAC class] platform design there is not the ability to leverage design, test and trial and market buying power to the same extent afforded to a non-Parent Navy platform such as has been the experience with the USN [United States Navy] Adelaide class FFG [Guided Missile Frigate] and RN [Royal Navy] Leander class DE [Destroyer Escort].36

An increase in required sustainment

2.38 To meet the recommendations of the Rizzo Review, there has been an increase in the number of maintenance tasks compared to the pre-Rizzo period. The 2015 advice to Deputy Chief of Navy reported that:

The number of URDEFs [Urgent Defects] have increased with an increase of 130 URDEFs from FY 13/14 to FY 14/15. There has been an increasing trend primarily on Priority 2 URDEFs with 525 raised in 2013, 669 raised in 2014 and 723 already raised in 2015 (to date 26 Oct 15).

2.39 Table 2.2 shows the reported Priority 1 and 2 Materiel Deficiency Reports from 2015–16 to 2017–18.37

Table 2.2: Reported Priority 1 and 2 Materiel Deficiency Reports, 2015–16 to 2017–18a,b

Financial year

Priority 1 Materiel Deficiency Report

Priority 2 Materiel Deficiency Reportb

 

Reports raised

Open at end of periodc

Reports closed

Reports raised

Open at end of periodc

 

2015–16

43

7

28

623

887

 

2016–17

39

6

16

708

1115

 

2017–18

23

2

22

603

810

 

Note a: The data set does not include those Materiel Deficiency Reports which have either been up- or down-graded.

Note b: Materiel Deficiency Reports were not raised for HMAS Perth (FFH-157) as it was placed into lay-up in 2017–18. This contributed to the reduction in reports raised in 2017–18.

Note c: ‘Reports closed’ for Priority 2 Materiel Deficiency Reports are not reported.

Note d: ‘Open at end of period’ are those materiel deficiency reports open at the end of relevant reporting month.

Source: ANAO analysis of monthly reported Sustainment Performance Reporting System results for Key Health Indicator ‘priority 1 materiel deficiency reports raised’ and ‘open priority 2 materiel deficiency reports’.

Extension of the life-of-type

2.40 The original planned withdrawal date was 2033 for the last ANZAC frigate. In June 2018, the Government was advised of Defence’s intention to extend the life-of-type of the ANZAC class, with the final ship to decommission in 2043 (as shown in Figure 2.2). Extending the life-of-type of the ANZAC class is intended to accommodate the design, build and introduction into service of Navy’s new Hunter class frigates under the Government’s 2017 Naval Shipbuilding Plan. Under the Plan, nine Hunter class ships will be built in Australia, with the first-of-class expected to enter service in 2027.38

Figure 2.2: Extension of the ANZAC class frigates’ life-of-type

A bar chart showing the ANZAC class frigates have had their life-of-type extended by 10 years from 2033 to 2043.

Source: ANAO analysis of Defence documentation.

2.41 The extension of the life-of-type of the ANZAC class frigates presents three significant risks for maintaining a surface ship capability during the transition from the ANZAC to Hunter class:

  • Maintaining the material state of the ANZAC class frigates’ hulls and sub-systems — which have been assessed to be in a degraded state (see paragraph 2.28).
  • Managing obsolescence within the ANZAC class — to maintain a contemporary level of capability and avoid a capability gap during the transition from the ANZAC class frigates to the Hunter class frigates. Further upgrades will be challenging as the ANZAC class is already considered to be at its design boundaries in terms of displacement and power generation (see paragraph 2.33).
  • As the class continues to age, the cost of sustainment and obsolescence management can be expected to increase (as noted in the 2011 Rizzo Review).

2.42 Defence’s June 2018 advice to the Government of its intention to extend the life-of-type of the ANZAC class, to accommodate the selection of the Hunter class as Navy’s new frigate, did not include an analysis of the ANZAC class’ physical capacity to deliver the required capability until 2043. This approach does not align with the need to undertake strategic planning across programs of work as envisaged by the 2016 Defence Integrated Investment Program:

Both Defence and Australian industry will have a heavy workload to deliver, upgrade and sustain Australia’s future maritime force. A challenge will be to successfully manage the transition between the existing and new submarine, frigate and patrol boat fleets, in particular ensuring the continued availability of required capabilities to meet the Government’s tasking. Strategic planning across programs of work over several decades, as opposed to the past project-by-project approach, will be essential in meeting this challenge.39

2.43 Defence is undertaking a transition study for the ANZAC class frigates addressing the life-of-type extension, which is due to be completed in late 2019. Defence has commissioned the WAMA to undertake the transition study.

2.44 Undertaking the transition study after the decision to extend has been made is not without risk. For example, in October 2018, the Australian Strategic Policy Institute released a report on the capability transition from the Collins class submarines to the future submarine fleet which points to risks involved in transitioning from one complex platform to another:

Governments generally expect there to be no decline in capability throughout transition, but that might not be easy. The outgoing platform is nearing, or often past, the end of its design life and, due to obsolescence, might not be able to deliver the quality or quantity of capability required (which is why it’s being replaced) … The incoming platform isn’t a sure bet in terms of capability either. The schedule for its entry into service, particularly for a complex, developmental platform, might not be reliable. This is even more probable many years ahead of its actual entry, when some key transition decisions may need to be made (such as the basing location, the ramp-down schedule for the old platform, and workforce development). Also, teething troubles may limit the level of capability it can provide for some time, requiring the old platform to stay in service even longer than originally planned. Moreover, if the new platform is a fundamental step change in capability, or has a significantly different operating model, a whole raft of new enablers (such as new infrastructure, information systems, and training and maintenance facilities) may need to be in place before it provides its full capability, finally allowing the old platform to retire.40

2.45 The extension of the life-of-type of the ANZAC class frigates will require ongoing investment by Defence to maintain the class’s capability and to address: the latent effects of what Defence acknowledges to be an under-funded and under-resourced sustainment function; and the effects of operating the ships outside their Statement of Operating Intent. The success of the Government’s Naval Shipbuilding Plan also relies on Defence’s ability to maintain the ANZAC class capability to accommodate the design, build and introduction into service of the Hunter class frigates.

2.46 Auditor-General Report No. 39 2017–18 Naval Construction Programs—Mobilisation, included the following recommendation:

That Defence, in line with a 2015 undertaking to the Government, determine the affordability of its 2017 Naval Shipbuilding Plan and related programs and advise the Government of the additional funding required to deliver these programs, or the Australian Defence Force capability trade-offs that may need to be considered.

2.47 Defence disagreed with this recommendation.41 The recommendation was made in the context of an evolving naval shipbuilding policy and investment landscape which had moved on from the 2016 Defence Integrated Investment Program’s cost assumptions. Continuing changes in the Defence policy landscape, such as the decision to extend the ANZAC class life-of-type, indicate that there would be benefit in reviewing the cost assumptions for ANZAC class sustainment.

Recommendation no.2

2.48 In the context of developing its transition plan for the ANZAC class life-of-type extension, Defence review the capital and sustainment funding required to maintain the ANZAC class frigate capability until 2043, and advise the Government of the funding required to meet the Government’s capability requirements for the class or the capability trade-offs to be made.

Department of Defence’s response: Agree.

3. Performance monitoring and reporting

Areas examined

This chapter examines whether the Department of Defence (Defence) has established an appropriate framework to monitor and report on the effectiveness and efficiency of operating the ANZAC frigates, in particular, whether relevant, reliable and complete performance measures had been developed and fit-for-purpose performance monitoring and reporting arrangements have been implemented.

Conclusion

Defence has established a performance framework for the ANZAC class frigates’ sustainment, with performance measures included in the Materiel Sustainment Agreement and reports provided to senior Defence leaders. While the performance measures adopted by Defence are relevant, the performance framework is not fully effective because the performance measures are:

  • only partially reliable — as targets and/or plans regularly change; and
  • not complete — as the measures do not address sustainment outcomes and efficiency.

In 2017–18 most of the Key Performance Indicators reported against were consistently not met.

The transparency of external reporting on the ANZAC frigates’ sustainment expenditure does not include Defence staffing costs or operational sustainment expenditure.

Areas for improvement

This chapter includes one recommendation aimed at improving the reliability and completeness of performance information for ANZAC class frigates sustainment.

Has Defence developed relevant, reliable, and complete performance measures for the sustainment of its ANZAC class frigates?

The performance measures adopted for the sustainment of the ANZAC class frigates are relevant but only partly reliable, as targets and/or plans regularly change. Further, the performance measures are not complete, as they do not address sustainment outcomes or efficiency.

3.1 The performance monitoring and reporting arrangements agreed by the Royal Australian Navy (Navy) and the Capability Acquisition and Sustainment Group (CASG) for the ANZAC class are set out in the ANZAC class Product Delivery Schedule of the Navy Materiel Sustainment Agreement. Consistent with Navy’s Materiel Sustainment Agreement Performance Framework (see Appendix 3), the 2018–19 Product Delivery Schedule required reporting and monitoring against four Key Performance Indicators, 14 Key Health Indicators and one Strategic Sustainment Analytic, as outlined in Table 3.1.42 For a description of the performance measures, see Appendix 3.

Table 3.1: Performance measures for the ANZAC frigates, 2018–19

Performance measure
Key Performance Indicators
Monthly Materiel Ready Daysa achievement
External maintenanceb period milestone achievement
Price reliability — year to date price achievementc
Conformance to operating intent
Key Health Indicators
Systems Program Office staffing levels
Capability Manager Representative staffing levels
Priority 1 materiel deficiency reports raisedd
Open Priority 2 materiel deficiency reportsd
External maintenance period cost growth
Organic level maintenancee backlog
External maintenance backlog
External maintenance period effectiveness
Demand satisfaction rate
External maintenance period demand satisfaction rate
NAVALLOWconfiguration effectiveness
Cannibalisationg events
Open variations
Open permanent engineering change proposals
Strategic sustainment analytic indicator
CASG cost per Materiel Ready Dayh

Note a: A Materiel Ready Day is any programmed day where a platform is not in an external maintenance period, undergoing defect repair, in extended readiness, or subject to an urgent defect that because of its nature prevents the ship from achieving its current tasking.

Note b: External maintenance is maintenance normally performed in port, by industry.

Note c: Reporting for this Key Performance Indicator is required individually against the following price segments:

Year to Date Product Price Achievement — Baseline;
Year to Date Product Price Achievement — Operations;
Year to Date Product Price Forecast — Baseline; and
Year to Date Product Price Forecast — Operations.

Note d: Navy assigns priorities to defects as follows:

Priority 1 — Safe: a safety related defect, condition or deficiency including information, data or documentation, that precludes: the ship remaining at sea or sailing.
Priority 1 — Operations: a defect, condition or deficiency, including information, data or documentation that prevents the ship or establishment from completing a specified or implied task.
Priority 2: a defect or condition that significantly limits seaworthiness, personnel safety or operational capability, but does not preclude scheduled operational activities; or that significantly increase the probability of not being able to complete any potential tasking; and requires rectification at the next suitable opportunity in the existing program.
Priority 3: a defect or condition that does not warrant classification as a priority one or two because an alternative engineering solution or significant redundancy exists. The defect places a significant burden on ship’s staff.

Note e: Organic level maintenance is maintenance normally performed on-board by Navy personnel.

Note f: NAVALLOW (Navy Allowance) is the Navy logistic system which records shipboard materiel support for installed and portable equipment fitted to Navy ships and establishments.

Note g: Cannibalisation is the process of removing a working part from one piece of equipment, such as a ship or submarine, to put it into another that is in greater operational need.

Note h: Calculated as the combined monthly product price baseline and operations actuals and the monthly Materiel Ready Days achieved. To derive the value for the current reporting period, the monthly cost per Materiel Ready Day is calculated for the current and previous 11 reporting periods, and then averaged.

Source: Department of Defence.

Changes to performance measures

3.2 In 2016, Navy and CASG conducted a joint review of Navy’s performance framework to validate that the design and practical application of the framework was consistent with the endorsed approach and to identify opportunities for improvement. Notably, two Key Performance Indicators were removed from the framework as part of the review:

  • ‘Cost per materiel-ready day achieved’ — removed on the basis that the measure required the Systems Program Office to enter four separate data elements and ‘stakeholder compliance’ was reportedly low.43This indicator became a Strategic Sustainment Analytic.
  • ‘Priority one materiel deficiency reports raised’ — removed on the basis that it duplicated data already reported for one Key Health Indicator (‘priority one materiel deficiency reports open’). The existing Key Health Indicator was renamed and restructured to ‘priority one materiel deficiency reports raised’ Key Health Indicator.

3.3 In mid-2017, a further Key Health Indicator ‘funding adequacy over Defence Management and Financial Plan’ was suspended. Whilst the Key Health Indicator is no longer reported against, it remains in the ANZAC class Product Delivery Schedule for 2018–19. In November 2018, the Department of Defence (Defence) advised that the metric had been suspended due to issues obtaining data.

Assessment of performance measures

3.4 Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements assessed Navy’s performance monitoring and reporting arrangements, including for the ANZAC class frigates:

Overall, Navy’s KPIs/KHIs [Key Performance Indicators and Key Health Indicators] are appropriately designed and measurable. The KPIs provide balanced and usable coverage of Navy’s sustainment products in terms of availability, cost, schedule, and materiel deficiencies.

While Navy’s performance measures are a step forward, there remain areas for improvement. None of Navy’s KPIs address outcomes, which is one of the four types of measures included in the DMO’s [now CASG] guidance. Measures such as Materiel Ready Days Achievement identify availability of platforms, but they do not indicate whether platforms were available when needed for operations. Further, some of the measures are not necessarily free from bias — that is, allowing for clear interpretation of results.44

3.5 Whilst there have been several changes to performance measures since 2015 (as outlined in paragraphs 3.2 to 3.3), the findings of this audit are consistent with the ANAO’s previous audit findings. The 2018–19 Key Performance Indicators and Key Health Indicators for the ANZAC frigates were assessed as relevant, partly reliable, and not complete (in terms of the program). Table 3.2 summarises the ANAO’s assessment of the Key Performance Indicators and Key Health Indicators for the ANZAC frigates. The strategic sustainment analytic indicator is not included in Table 3.2 as it does not have an established target or require formal review and signoff.

Table 3.2: Assessment of 2018–19 Key Performance Indicators and Key Health Indicators for the ANZAC frigates

Characteristica Assessment
Relevant Relevant. Each of the measures is designed to measure relevant outcomes for the sustainment of the ANZAC frigates.
Reliable Partly reliable. Some of the measures are not necessarily free from bias with:

  • targets for nine of the 14 Key Performance Indicators based on performance against a short-term rolling averageb; and
  • targets for Key Performance Indicators ‘monthly Materiel Ready Days achievement’ and ‘price reliability–year to date price achievement’ subject to regular in-year changes in plans or targets.

Targets set in this way may potentially mask deterioration and/or improvements in performance and not allow for clear interpretation of results.

Complete Not complete because:

  • there is no measure of outcomes (for example, rate of effort achievement, mission capable days achievement); and
  • there are no measures of efficiency.

Note a: These characteristics are based on the criteria developed to evaluate the appropriateness of an entity’s Key Performance Indicators contained in Auditor-General Report No.33 2017–18 Implementation of the Annual Performance Statement Requirements 2016–17.

Note b: The short-term rolling average is an average of the last three monthly outcomes, exclusive of null values.The nine Key Health Indicators are: priority 1 materiel deficiency reports raised; open priority 2 materiel deficiency reports; organic level maintenance backlog; external maintenance backlog; external maintenance period effectiveness demand satisfaction rate; external maintenance period demand satisfaction rate; cannibalisation events; open variations; and open permanent engineering change proposals.

Source: ANAO analysis.

3.6 As noted in Table 3.2, the key performance measures are only partly reliable as some of the measures are not free from bias (that is, the interpretation of results is not clear):

  • The majority of Key Health Indicators listed in Table 3.1 (see paragraph 3.1) report against the short-term rolling average and not a fixed target. The short-term rolling average is based on the results of the last three months and can change. Regular changes in the short-term rolling average can mask longer-term changes in performance, as targets are adjusted up or down in line with changes in performance over time. For example, there were instances where similar performance outcomes received different traffic-light ratings and different performance outcomes received similar traffic-light ratings.
  • The Key Performance Indicators for ‘monthly Materiel Ready Days achievement’ and ‘price reliability—year to date price achievement’ are expressed in percentage terms against a plan or target, and changes in plans or targets may mask deteriorating performance. In 2017–18 there were multiple changes to the baseline and operational expenditure estimates.

3.7 As noted in Table 3.2, Navy’s Key Performance Indicators and Key Health Indicators do not address sustainment outcomes. Whilst the performance measure such as Materiel Ready Days achievement identifies the availability of the platforms, it does not indicate whether platforms were able to meet operational requirements.

3.8 As noted in Table 3.2, Navy’s Key Performance Indicators and Key Health Indicators do not address sustainment efficiency. The CASG cost per Materiel Ready Day’ strategic sustainment analytic is described by Defence as a ‘broad measure of relative cost efficiency associated with the delivery of Materiel Ready Days’.45However, the measure: has no set target and is not benchmarked, limiting the consistent measurement, assessment and interpretation of results; and does not require formal review or signoff, limiting management oversight of performance. The ANAO has previously observed that this key performance measure is not useful in determining the total cost of the capability to Defence.46

3.9 The Key Performance Indicators identified in the Materiel Sustainment Agreements (listed in Table 3.1), are not aligned with the performance indicators included in the Warship Asset Management Agreement. This is discussed further in Chapter 4.47

Recommendation no.3

3.10 Defence review the key performance measures for the ANZAC class frigates’ sustainment to ensure they are reliable and complete.

Department of Defence’s response: Agree with qualification.

3.11 This will require consultation across Defence to review extant reporting metrics; as these are currently standardised across multiple warship classes.

Has Defence established fit-for-purpose performance monitoring and reporting arrangements?

Defence has established arrangements to monitor and report on the sustainment of the ANZAC class frigates, with senior Defence leaders made aware of the sustainment risks and issues experienced by the ANZAC class. The performance reporting indicates that there was underperformance for most of the Key Performance Indicators for the sustainment of the ANZAC class frigates during 2017–18. External reporting on the ANZAC class frigates’ sustainment expenditure would be more transparent if it included Defence staffing costs and operational sustainment expenditure.

Internal reporting

Sustainment Performance Management System

3.12 The Product Delivery Schedule requires performance reporting for the ANZAC class to occur via the Sustainment Performance Management System (SPMS). SPMS is Defence’s primary sustainment reporting and performance management system.48 SPMS can include Key Performance Indicators and Key Health Indicators (as defined by the Services), and also strategic sustainment analytics used for cross-platform performance analysis.49 The performance outcomes in SPMS are reviewed and signed off monthly by representatives from both Navy and CASG. Subsequently, performance outcomes reported in SPMS for Navy Product Delivery Schedules are included in a monthly brief from CASG to the Deputy Chief of Navy, and used to inform the Quarterly Performance Report to senior stakeholders including the Defence Minister (see paragraph 3.22).

3.13 Performance reporting against targets is supported by commentary and review from representatives of CASG and the Capability Managers. The commentary includes relevant discussion on current issues, remediation, future risks, and areas requiring further attention. For example, Defence internal reporting captured the implications of placing HMAS Perth in extended lay-up50 (as shown in Figure 3.1), including reduced workforce pressures on the rest of the ANZAC class frigates and impacts on Surface Combatant availability. Defence advised the Minister for Defence of the decision to place HMAS Perth (FFH-157) in lay-up in December 2017.

Figure 3.1: HMAS Perth (FFH-157) in lay-up at the Henderson shipyard — July 2018

A photograph of the ANZAC class frigate HMAS Perth in lay-up at the Henderson Shipyard.

Source: ANAO site visit to Henderson shipyard — July 2018.

3.14 The unplanned, extended lay-up of HMAS Perth places further pressure on the other ANZAC class frigates and potentially adds to the cycle of operating the class outside of its Statement of Operating intent to meet capability and availability requirements.

3.15 As at 2 October 2018, three of Navy’s eight ANZAC class frigates were in dry-dock at the Henderson shipyard — HMAS ANZAC (FFH-150) and HMAS Arunta (FFH-151) were in deep-cycle maintenance and HMAS Perth (FFH-157) was in lay-up. Figure 3.2 shows the ANZAC class frigates’ Materiel Ready Days targets for 2018–19.

Figure 3.2: ANZAC class frigates’ target Materiel Ready Days 2018–19

Note a: A Materiel Ready Day is any programmed day where a platform is not in an external maintenance period, undergoing defect repair, in extended readiness, or subject to an urgent defect that because of its nature prevents the ship from achieving its current tasking.

Source: ANAO analysis of Defence documentation.

Reporting on sustainment for the ANZAC class frigates for 2017–18

3.16 Table 3.3 provides a summary of reported performance for Key Performance Indicators and Key Health Indicators, as recorded in the Sustainment Performance Management System for 2017–18. The strategic sustainment analytic indicator is not included in Table 3.3 as it does not have an established target or require formal review and signoff.

Table 3.3: Performance against Key Performance Indicators and Key Health Indicators as recorded in the Sustainment Performance Management System 2017–18a

Performance measure

2017

2018

J

A

S

O

N

D

J

F

M

A

M

J

Key Performance Indicators
Monthly Materiel Ready Days achievement

External Maintenance Period Milestone Achievement

Conformance to operating intent

Year to date price achievement — baseline

Year to date price achievement — operations

Year end product price forecast — baseline

Year end product price forecast — operations

Key Health Indicators
Systems Program Office staffing levels

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

N

▲(w)

▲(w)

▲(w)

▲(w)

Capability Manager Representative staffing levels

N

N

N

N

N

N

N

N

N

▲(w)

N

Priority 1 materiel deficiency reports raised

▲(w)

▲(w)

N

▲(w)

Open Priority 2 materiel deficiency reports

▲(w)

▲(w)

N

▲(w)

External maintenance period cost growth

N

N

N

N

N

N

N

Organic level maintenance backlog

▲(w)

▲(w)

▲(w)

External maintenance backlog

N

▲(w)

▲(w)

External maintenance period effectiveness

▲(w)

▲(w)

N

▲(w)

N

N

N

N

Demand satisfaction rate

▲(w)

N

▲(w)

▲(w)

▲(w)

▲(w)

External maintenance period demand satisfaction rate

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

NAVALLOW configuration effectiveness

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

N

▲(w)

▲(w)

▲(w)

Cannibalisation events

▲(w)

N

▲(w)

Open variations

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

N

Open permanent engineering change proposals

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

▲(w)

KEY:

◆ Green  ▲ Amber  ▲ (w) White

■ Red  – not reported

N value unable to be obtained for current period, or intended to be reported by the end of a reporting period

Note a: SPMS uses one traffic-light rating system for Key Performance Indicators — ‘red’, ‘amber’ and ‘green’ — and a different one for Key Health Indicators — ‘red’, ‘white’, ‘green’ and ‘N’. The ANAO has shown the ‘white’ indicators as ‘amber’ with a ‘(w)’ to allow for ease of comparison between performance measures. See Appendix 3 for the tolerance thresholds for each performance measure. There is no established target or reporting thresholds for the strategic sustainment analytic indicator.

Source: SPMS.

3.17 As shown in Table 3.3 by the indicators identified as ‘red’ and ‘amber’, Defence reported underperformance against most Key Performance Indicators for the sustainment of the ANZAC class frigates in 2017–18.

3.18 As also shown in Table 3.3, several Key Health Indicators were regularly rated as ‘N’ value (that is, value unable to be obtained for current period, or intended to be reported by the end of a reporting period). In November 2018, Defence advised that ‘External maintenance period cost growth’ and ‘External maintenance period effectiveness’ are reported as ‘N’ in months where relevant stage gate reports or external maintenance periods are not due or completed. Defence advised of two incidences for ‘External maintenance period cost growth’ where the ‘N’ value was reported due to growth being outside of the reporting threshold for overspends and underspends. Defence further advised that ‘Capability Manager Representative staffing levels’ is reported as ‘N’ as it is ‘not considered to be a useful metric and is not tracked’.

Fleet screenings

3.19 Navy also regularly reviews the status of its fleet through its biannual ‘fleet screenings’. These reviews consider the performance of Navy’s platforms, and provide an opportunity for the Deputy Chief of Navy to manage sustainment funding between different Product Delivery Schedules, taking into account changing circumstances and operational needs. The reviews occur in February/March and September/October each year. In 2014, Navy internal advice noted that:

Cognisant of the findings of the Rizzo Report, the focus of the reviews has evolved beyond their original financial emphasis into a forum for DCN [Deputy Chief of Navy] and the relevant DMO [Defence Materiel Organisation] Division/Group Head to consider how their respective organisations are meeting their obligations for whole of life management and sustainment of Navy capability. Financial planning and performance remains a key element of the reviews, and the composite view of Navy sustainment pressures provided by the biannual review activity enables DCN [Deputy Chief of Navy] to make informed capability and resource allocation decisions.

3.20 The results of recent Fleet Screenings were examined in paragraphs 2.9 to 2.10 of this Report.

Quarterly reporting to Ministers

3.21 The performance of the sustainment of the ANZAC class frigates is also included in Defence’s Quarterly Performance Report, which is provided to senior stakeholders within Government and Defence. In relation to sustainment, the report provides information on availability, price achievement, and information on the utilisation of Australian industry for different Defence projects. The data included in the report regarding availability and price achievement is compiled from the results reported in the Sustainment Performance Management System.

3.22 The sustainment of the ANZAC fleet was listed as a Product of Interest in the Quarterly Performance Report from June 2016 to December 2016.51 In November 2018, Defence advised the ANAO that:

CN02 [ANZAC class sustainment] was placed on the Products of Interest list in Jun[e] [20]16 primarily due to the risk to capability resulting from delay in [HMAS] Arunta’s Anti-Ship Missile Defence Upgrade program and the obsolescence of class-wide major ships’ systems. It remained on the Products on Interest list in Dec 16 due to funding pressures. In Q3 [Quarter 3] of FY [Financial Year] 2016/2017 these funding pressures were rectified by way of a Major Surface Ships (MSS) cross levelling activity and consequently the Product was removed from the list.52

3.23 Defence further advised in February 2019 that:

CN02 [ANZAC class frigates sustainment] was first reported in the June 2016 Quarterly Performance Report, as a Product of Interest. Reasons behind CN02 becoming a [Product of Interest] was due to issues derived from a number of sources including the transition to the new support contract, continued difficulty in getting ships out of Anti-Ship Missile Defence program and ongoing issues caused by obsolescence of some of the equipment. CN02 was last featured in the December 2016 Quarterly Performance Report and was removed from the Product of Interest list for the March 2017 Quarterly Performance Report with the following criteria: A revised Statement of Operating Intent (SOI) was published to articulate the process to be followed for SOI variances, with the FFH [ANZAC class frigates] force assignment to the Middle Eastern Region to be covered under this provision. The Operational Funding cost capture model was refined to more accurately reflect cost of operating ships outside of the SOI. This proposal achieved no win/no loss funding for increased through life support costs driven by increase and/or alternate usage of platform.

External reporting

3.24 The ANZAC class frigates are a ‘Top 30’ sustainment product and information on sustainment expenditure is publicly reported in the Defence Portfolio Budget Statements, Portfolio Additional Estimates Statements and Annual Reports.

3.25 Since 2007–08 Defence has reported financial and descriptive information for the ‘Top 20’ or ‘Top 30’ sustainment products (‘Top 20’ to 2012–13 and ‘Top 30’ from 2013–14). In general, reporting has included: a description of sustainment activity; estimated expenditure; actual expenditure; the variation between estimated and the actual expenditure; and some explanation of the variation between estimates in the Portfolio Budget Statements and Portfolio Additional Estimates Statements. Sustainment expenditure for the ANZAC frigates reported externally is not complete, as it does not include Defence staffing costs or operational sustainment expenditure.53

4. Administration of the sustainment strategic partnership

Areas examined

This chapter examines whether the Department of Defence (Defence) effectively administers the ANZAC sustainment strategic partnership to achieve specified availability and performance outcomes. It considers the sustainment arrangements recently established with industry and key performance measures.

Conclusion

Defence entered into a sole sourced alliance contract with its existing industry partners, without a competitive process.

It is too early to assess the effectiveness of Defence’s administration of the new contracting arrangements, known as the Warship Asset Management Agreement, which took full effect in January 2018 after an 18-month transition period. Defence’s regular internal performance reporting and monitoring does not capture the performance of the Agreement.

Areas for improvement

This chapter includes two recommendations aimed at improving: internal Defence guidance on when sustainment programs should be referred to the Defence Investment Committee; and the alignment between performance measures reported on internally and those in the Warship Asset Management Agreement.

Has Defence established effective contracting arrangements to sustain the ANZAC class frigates?

Defence entered into a sole sourced alliance contract (the Warship Asset Management Agreement) with its existing industry partners, under an exemption from the Commonwealth Procurement Rules.

In the absence of a competitive process, Defence determined that value-for-money had been achieved after considering cost, the expertise of the industry partners, and their previous experience in sustaining the ANZAC class.

It is too early to assess the effectiveness of the contracting arrangements for ANZAC class sustainment, which took full effect in January 2018 after an 18-month transition period. The strategic partnership arrangement is expected to: drive efficiency; transfer risk to industry; reduce Defence’s cost of ownership; simplify contract administration; and reduce contract disputes. However, the arrangements may reduce Defence’s leverage over industry participants.

Defence entered into the new sustainment contract without seeking endorsement from the Defence Investment Committee or the Minister for Finance, on the assumption that ANZAC class sustainment had been approved at the time of the ships’ acquisition in the 1980s or possibly when they were introduced into service in the 1990s. Defence should have sought advice from central agencies on the most appropriate handling of this matter, given the high value of this procurement and the uncertainty over past approvals.

Warship Asset Management Agreement

4.1 Since 1998, the Department of Defence (Defence) has outsourced the sustainment of the ANZAC class to industry through a series of contractual arrangements administrated by the ANZAC Systems Program Office. Contracted sustainment arrangements are outlined in Appendix 4.

4.2 Since July 2016, sustainment activities for the ANZAC class have been delivered through the Warship Asset Management Agreement (the WAMA or the Alliance). The WAMA is an alliance between the Commonwealth of Australia (the Commonwealth), BAE Systems Australia (BAE Systems), Saab Australia (Saab) and Naval Ship Management (Australia) (NSM). The Alliance is intended to be:

an outcomes based commercial model with all parties sharing in the risk and opportunities associated with delivering improved performance and cost savings in delivering both capability acquisition and sustainment outcomes for the ANZAC Class.54

Establishment of the Warship Asset Management Agreement

4.3 The WAMA was developed over a period of two and a half years (as illustrated in Figure 4.1) in the following phases:

  • Deed of Commitment — agreement between the existing service providers — the ANZAC Ship Integrated Materiel Support Program Alliance Master Agreement (BAE and Saab) and the Group 3 Group Maintenance Contract provider (NSM) — to enter into a new alliance agreement in July 2015;
  • Program Agreement 1 — the transition phase from the previous alliance arrangement (July 2016 to December 2017); and
  • Program Agreement 2 — operation of the WAMA sustainment enterprise (January 2018 to December 2022).

4.4 The Alliance has continued to develop as the roles and responsibilities of the participants to the Agreement have been refined. For example, the signing of contracts for sustainment equipment procured through the Military Integrated Logistics Information System was the responsibility of the Commonwealth under the old arrangements but under the new arrangements that responsibility lies with the Alliance participants through an ‘Authority to Act Instrument’. At the inception of the WAMA, it was not contractually clear which participant was responsible.

Figure 4.1: Warship Asset Management Agreement development timeline

A timeline of the development of the Warship Asset Management Agreement from the signing of the Deed of Commitment on 10 July 2015 to the Commencement of the Warship Asset Agreement 2 on 1 January 2018. The timeline also includes the construction of the A

Source: ANAO analysis of Defence documentation.

Deed of Commitment

4.5 The Deed of Commitment was signed on 10 July 2015 and committed the Commonwealth, BAE Systems, Saab and NSM to the development of a new alliance agreement. Defence records indicate that the new alliance agreement was to be based on the principles of International Standards Organisation (ISO) 55000 Asset Management.55 The adoption of the ISO Asset Management principles as the basis for the Alliance structure was intended to address the findings of the 2011 Rizzo Review. The review found that Navy’s approach to asset management was informal and that Defence lacked discipline and rigour in the management of maritime assets.56

4.6 Defence internal advice (dated December 2015) stated that the WAMA model was also consistent with findings made in the First Principles Review Creating One Defence (2015) because the Alliance would enable Defence to focus on governance and planning functions relating to the sustainment of the ANZAC class frigates rather than the management and delivery functions.57 Defence identified the following benefits from the WAMA:

  • Alignment to the Capability Acquisition and Sustainment Group Assurance Model.
  • The transfer of risk and opportunity to industry.
  • The potential to drive efficiency in the sustainment operation.
  • A reduction in contractual transactions between Defence and industry.
  • Certainty for industry to invest in the sustainment operation.
  • Better opportunity to reduce the cost of ownership through a strategic partnership with industry aligned with performance outcomes.
Endorsement to proceed to tender

4.7 In December 2015, Defence endorsed a ‘limited tender collaborative procurement process’ to establish the WAMA. The process was limited to the providers of sustainment to the ANZAC class frigates under the previous sustainment arrangements — BAE Systems, Saab and NSM. Advice to the delegate stated that:

A strategic partnership with current industry participants, BAE, NSM and Saab, offers the least risk and maximum opportunity in ensuring the capability and capacity is available to support the Commonwealth requirements …

The Secretary, as the Accountable Authority for Defence, has pre-determined, and published in the DPPM [Defence Procurement Policy Manual] at chapter 1.2, that procurements for the design, development, integration, test, evaluation, maintenance, repair, modification, rebuilding and installation of military systems and equipment are exempt from Division 2 of the CPRs [Commonwealth Procurement Rules]. Therefore, the current service providers can be engaged through a limited tender exempt from the Application of Division 2.58

4.8 Notwithstanding this exemption from Division 2 of the Commonwealth Procurement Rules, the Secretary of the Department of Defence has determined in the Defence Procurement Policy Manuals (2014 and 2017) that all procurements are required to demonstrate value for money, which is the core rule of the Commonwealth Procurement Rules.59

Program Agreement 1

4.9 Program Agreement 1, commonly referred to within Defence as the ‘transition’, was signed on 30 June 2016 between Defence, BAE Systems, Saab, and NSM. The agreement outlined measures of performance as well as estimated costs.

4.10 During the Program Agreement 1 phase, in addition to continuing the sustainment of the ANZAC class frigates, Defence and the industry partners (through the WAMA’s Alliance Management Office) established the WAMA’s frameworks and systems, and developed cost estimates for sustaining the ANZAC class frigates during the Program Agreement 2 phase.

Program Agreement 2 and approval processes

4.11 A cost proposal for Program Agreement 2 was developed by industry partners and Commonwealth personnel within the Alliance Management Office.60 In September 2017, Defence rejected the proposal because it:

…was not in accordance with Commonwealth Guidance, being $277 million over the provided guidance. The AMO [Alliance Management Office] insisted that [the proposal] represented the amount required to achieve the levels of performance achieved in the Transition Program Agreement [Program Agreement 1] and that [the proposal] represented Value-for-money for the Commonwealth.

4.12 The Alliance Management Office refined the cost proposal by transferring up to $100 million of maintenance to Navy’s Fleet Support Unit, and further reducing the scope of the proposal to meet Defence’s budget guidance. Defence determined that the refined proposal offered value-for-money on the basis of scope, schedule, budget, risk and contingency. A submission to the delegate dated 18 December 2017 further advised that:

… Use of the WAMA ensures that the Commonwealth utilises the domain and specialised skills retained by the AIPs [Australian Industry Partners], and further achieves VFM [value-for-money] as the AIPs [Australian Industry Partners] have a proven record of achievement on the ANZAC Class.

The AIPs [Australian Industry Partners] are Australian companies and the Alliance Activities will provide an overall benefit to Australian interests through sustaining the ANZAC class warships and developing local workforce and capabilities.

4.13 Through the submission, the delegate was asked for approval under section 23 of the Public Governance, Performance and Accountability Act 2013 for funding of $1.4 billion for a five-year period (1 January 2018 to 31 December 2022). The costing outlined in the submission was based on the results of the ‘limited tender collaborative interactive procurement process’ endorsed by the delegate two years prior. The delegate was advised that the cost initially proposed by the Alliance had been refined during the procurement process and assessed by Defence as offering value-for-money at an acceptable level of risk. The delegate approved the requested funding of $1.4 billion on 19 December 2017.

4.14 The submission to the delegate further advised that:

Notification was received … from the Chief Finance Officer Navy (CFO-N) office and Vice Chief of the Defence Force (VCDF) that this submission does not require consideration by the Investment Committee and/or the Minister of Finance, as the known scope of work has been previously approved.

4.15 The ‘notification’ referenced in the December 2017 submission occurred through an exchange of emails between Navy and the Capability Acquisition and Sustainment Group. On 14 June 2017, the Director of Financial Management of Submarines advised the Director-General of Maintenance of Surface Ships that:

Department of Finance have advised that if the spending has already been considered in an NSC [National Security Committee of Cabinet] process (such as a submission on an acquisition with NPOC [net personnel and operating costs]), then MINFIN’s [Minister for Finance] separate agreement is not needed. As WAMA sustainment would have been part of the CABSUB [Cabinet Submission] then we won’t need to go to MINFIN [Minister for Finance].

We’ve also been successful in not having to go to the Investment Committee for an extension to the Submarines In Service Support Contract, so we’re following up to see if the WAMA could be exempt as well. In order to do this I’d appreciate your advice on whether the sustainment aspects of ANZAC have previously been considered by NSC [National Security Committee of Cabinet]. Any other information you have which would support us seeking exemption from the IC [Investment Committee] would be appreciated.

4.16 On 14 June 2017, the Director-General of Major Surface Ships responded that:

I can’t really say whether ANZAC went to NSC [National Security Committee of Cabinet]. If it did it would have been in the [19]80’s when the project was initiated (assuming that sustainment funds were sought at the same time as the acquisition). Failing that it would have gone to NSC [National Security Committee of Cabinet] around the first delivery of a ship in 1996.

If it went to an NSC [National Security Committee of Cabinet] it would have gone to NSC [National Security Committee of Cabinet] at around the same time [as] Collins, which was a companion project to ANZAC.

4.17 On 17 June 2017, the Director of Financial Management of Submarines advised the Director-General of Maintenance of Surface Ships that:

The advice we have received is that if ANZAC has been through the NSC [National Security Committee of Cabinet] or the previously endorsed approval processes then we won’t need MINFIN [Minister for Finance] approval. I would suspect that such a large Project would have been through a rigorous approval process such as the precursor to NSC [National Security Committee of Cabinet].

The advice from VCD [Vice Chief of the Defence Force] was very clear that you don’t have to go to IC [Investment Committee], and I understand your agenda item has been cancelled.

4.18 Defence records indicate that the decision to proceed with WAMA Program Agreement 2, without seeking endorsement from the Defence Investment Committee or the Minister for Finance, was based on the assumption that ANZAC class sustainment had been approved with the ships’ acquisition in the 1980s or possibly with the introduction into service of the first ship in the 1990s.61 In the absence of evidence of necessary approvals, Defence should have sought advice from the central agencies on the most appropriate handling of the matter, bearing in mind the high value of this non-competitive procurement, the potential benefits of scrutiny within government of the value-for-money of the proposed sustainment agreements, and the current sustainment challenges faced by Defence in maintaining the ANZAC platform.62

4.19 There is no guidance within the Defence Capability Life Cycle Manual or the Defence Procurement Policy Manual as to when a current sustainment program should return to the Defence Investment Committee when there is uncertainty about the funding parameters originally agreed by Government. Where there is a risk that current funding decisions may not align with the original approvals, the Defence Investment Committee should consider proposals to establish or amend a sustainment program.

SEA 5000 Program

4.20 In late 2017, at the time that sustainment of the ANZAC class under the WAMA Program Agreement 2 was being considered and approved by the Delegate, the tender evaluation for the Hunter class (through the SEA 5000 program) was also being finalised. The ANZAC class sustainment program is significantly affected by the acquisition of the Hunter class (for example, the timing of their entry into service requires attention so as to avoid any capability gap). However, ANZAC class sustainment needs did not inform the SEA 5000 decision and Ministers were not advised of the cost of extending the ANZAC class’ life-of-type to 2043 or of the material state of the ANZAC class frigates, notwithstanding the strategic planning approach envisaged in the 2016 Defence Integrated Investment Program. As noted in paragraph 2.43, a strategic approach was expected to avoid ‘the past project-by-project approach’ in delivering, upgrading and sustaining the maritime force.

Recommendation no.4

4.21 To align with the strategic planning approach outlined in the Defence Integrated Investment Program, Defence develop guidance in the Capability Life Cycle Manual on when a proposal to establish or amend a sustainment program should be provided to the Defence Investment Committee and the Minister for Finance for consideration.

Department of Defence’s response: Agree with qualification.

4.22 The CLC [Capability Lifecycle Manual] already includes considerable guidance in the “Phase 4 – Sustainment and Disposal” chapter including Annex C on roles and responsibilities in regard to establishing or amending a sustainment program and the mechanisms through which this is done including reference to the Enterprise Business Committee, the Investment Committee, and when Ministerial or Government approvals are required.

4.23 At paragraph 14.3.5 the CLC explicitly states that where an upgrade is planned, budgeted, and approved at Gate 2, it should be able to proceed based on internal approval pathways should it remain within the broad parameters originally agreed by the Government.

4.24 The CLC is principles based and therefore Resource Management guides, Budget Process Operational Rules and Governance Arrangements are more appropriately placed to detail any changes required to business processes or thresholds/triggers.63

Conditions of Program Agreement 2

4.25 The WAMA compensation framework consists of three payment types:

  • reimbursable costs (‘limb 1’) — payment to reimburse the Alliance Industry Participants’ operating costs (for example, replacement parts for the frigates and shipyard personnel wages);
  • fees (‘limb 2’) — payment in addition to reimbursable costs that allow Alliance Industry Participants to make a profit, consistent with the Alliance achieving, but not exceeding, the agreed performance; and
  • pain-share and gain-share costs (‘limb 3’) — payments of gain-share by the Commonwealth to the Industry Participants, or payments of pain-share by the Industry Participants to the Commonwealth, to reflect an agreed sharing of the gain/pain, where the actual performance of the Alliance is superior/inferior to agreed targets in cost and other Key Performance Indicators.

4.26 Under this framework, there is a limitation on the Alliance Industry Participants’ liability for pain-share, with the pain-share between the Commonwealth and the Alliance Industry Participants capped at the fixed amount in ‘limb 2’. Any additional pain-share above this capped amount is attributable to the Commonwealth. There is no limit on the ‘gain-share’ payments by the Commonwealth to the Alliance Industry Participants.

4.27 As part of the WAMA, the Alliance participants have agreed that there ‘will be no litigation or arbitration between ourselves arising out of or in connection with this Agreement’ (excluding some legal rights). This approach may reduce Defence’s leverage over the Industry Participants.

4.28 An additional consideration relates to the designer and builder of the new Hunter class frigates (BAE Systems Australia) also being a key member of the ANZAC class frigates sustainment Alliance. In transitioning from the ANZAC class to the Hunter class frigates, Defence’s commercial leverage over BAE Systems for any cost or schedule overruns in the Hunter class design and build, will be limited due to the existing commercial relationship with BAE Systems to sustain the ANZAC class. For example, any liquidated damages applied to BAE Systems for underperformance on the Hunter class program, may be offset by the benefit received by BAE Systems through the extended sustainment of the ANZAC class. The ANAO found no arrangement within the WAMA contract to deal with this issue.

Reliance on Navy’s operational sustainment resources

4.29 The sustainment arrangements with industry under WAMA Program Agreement 2 requires Navy’s Fleet Support Unit to undertake $60 million of ANZAC sustainment work, with a desired output of $100 million over the life of the agreement.64 The value-for-money assessment conducted as part of the WAMA procurement process identified that a major risk to the ANZAC sustainment program meeting its performance requirements under Program Agreement 2 was the capacity of Navy’s Fleet Support Unit to meet its sustainment requirements.

4.30 ANAO discussions with Navy personnel during this audit indicated that the Fleet Support Unit’s capacity to assist with ANZAC sustainment is often limited, for the reasons outlined in Auditor-General Report No.17 2014-15 Recruitment and Retention of Specialist Skills for Navy:

The sailors needed to ensure a ship can go to sea often have the skills to undertake deeper maintenance of platforms ashore. The sailors the ANAO spoke to noted that these sailor postings sometimes led to delays in completion of maintenance jobs allocated to Fleet Support Units, usually by the Defence Materiel Organisation’s (DMO’s) Systems Program Offices [now Capability Acquisition and Sustainment Group]. The sailors considered that, as a result, DMO viewed Fleet Support Units as unreliable and, at times, preferred the reliability of a civilian contractor instead.65

Is the performance of the ANZAC sustainment strategic partnership reported on and is the partnership achieving contracted availability and performance outcomes?

Defence’s regular internal performance reporting and monitoring does not capture the performance of the Warship Asset Management Agreement. The current misalignment of performance measures in the Warship Asset Management Agreement with the framework set out in the ANZAC class Product Delivery Schedule of the Materiel Sustainment Agreement may result in a lack of clarity around the achievement of outcomes.

Defence’s initial assessment of the performance of the Warship Asset Management Agreement indicates that all measures had been met or exceeded as at late 2017. Defence plans to evaluate the value-for-money of its contracting arrangements in 2020.

Performance measures

4.31 The WAMA includes six key performance indicators.66 As indicated in Table 4.1, Defence has entered into a contract with performance measures that do not align with the Key Performance Indicators and Key Health Indicators in the ANZAC class Product Delivery Schedule which are monitored and reported on internally by Defence. For example:

  • the methodologies, reporting timeframes and performance targets used for half of the WAMA Key Performance Indicators do not align to the ANZAC class Product Delivery Schedule performance measures;
  • the WAMA has a performance measure relating to ‘utilisation of Fleet Support Unit’ which is not measured and reported on internally by Defence; and
  • two Key Performance Indicators the WAMA reports against — ‘cost per Materiel Ready Day achieved’ and ‘priority 1 urgent defects raised’67 — are classified as a strategic support analytic and Key Health Indicator, respectively, rather than key performance indicators in the Materiel Sustainment Agreement.

Table 4.1: Alignment of the current Warship Asset Management Agreement performance measures to the ANZAC class Product Delivery Schedule performance measures

Performance Measure

Alignment to the Product Schedule performance measures

Materiel Ready Days achieved

External maintenance period stage gate achievement

Cost per Materiel Ready Day achieved

Price reliability

Number of priority 1 urgent defects raised

Utilisation of fleet support

Source: ANAO analysis of Defence documentation.

4.32 It is important that performance indicators in the WAMA contract align to the performance indicators set out by Navy in the ANZAC class Product Delivery Schedule. For example, the misalignment of performance measures between the ANZAC class Product Delivery Schedule and the WAMA may result in a lack of clarity around priorities. Further, the WAMA has legal custody of ships undergoing sustainment in dry dock, Navy relies on reporting of the WAMA’s performance for insights into the achievement of agreed sustainment outcomes.

Recommendation no.5

4.33 Defence refine its performance reporting and management arrangements for the ANZAC class frigates by aligning Key Performance Indicators in the Warship Asset Management Agreement and those in the ANZAC class Product Delivery Schedule of the Navy Materiel Sustainment Agreement.

Department of Defence’s response: Agree.

Performance of the Warship Asset Management Agreement

4.34 Performance of the WAMA was assessed in late 2017 during Program Agreement 1 (the transition program agreement). That assessment informed the decision to proceed to Program Agreement 2. The assessment found that ‘the Alliance is progressing towards a sustained and consistent middle ground of performance, with improved execution occurring as the Alliance matures’. More specifically, the assessment found that the Alliance was:

  • meeting or exceeding the required performance level for all the Key Performance Indicators, except for ‘cost per Materiel Ready Day achieved’ which was not measured for Program Agreement 1 (see Table 4.2);68
  • meeting or exceeding the required performance level (representing the ‘business as usual’ benchmark) for all Key Health Indicators;69 and
  • offering an acceptable level of performance in the short term but will be unsatisfactory in the medium or longer term for all but one of the Strategic Performance Measures, which was assessed as meeting the required performance level.70

Table 4.2: Key Performance Indicators assessment for Warship Asset Management Agreement Program Agreement 1.

Key Performance Indicator Assessment for Program Agreement 1a
Materiel Ready Day achievement Met the required performance level
External maintenance period stage gates achievement Met the required performance level
Cost per Materiel Ready Day achieved Not Measuredb
Price reliability Exceeded the required performance level
Number of Priority 1 materiel deficiency reports raised Exceeded the required performance level
Utilisation of Fleet Support Unit Met the required performance level

Note a: There are four performance bands for Key Performance Indicators:

Performance Band 1 — level of performance exceeds the required performance level for respective Key Performance Indicator.
Performance Band 2 — level of performance meets the required performance level, better than minimum conditions of satisfaction.
Performance Band 3 — level of performance that may be tolerable for a short term but unsatisfactory in the medium or longer term because of the diminished value of the services, supplies and work.
Performance Band 4 — levels of performance where the value of services, supplies and work is considered to be negligible because the Commonwealth’s ability to attain the required outcomes is significantly affected.

Note b: Key Performance Indicator not assessed as part of Program Agreement 1.

Source: Defence.

4.35 As Program Agreement 2 began in January 2018, an assessment against Key Performance Indicators is yet to occur.

Evaluation of value-for-money under the Warship Asset Management Agreement

4.36 The first value-for-money assessment under the WAMA is not required until 2020; however, Defence undertook a value-for-money assessment in late 2017 as part of the transition to Program Agreement 2.71Defence concluded that ‘PA2 [Program Agreement 2] and its associated TCE [Target Cost Estimate] is considered to offer value-for-money at an acceptable level of risk’. As part of the value-for-money assessment, advice from the Alliance to Defence noted ‘significant KPI [Key Performance Indicator] performance impacts as a result of constraining the final budget to current available budget’.

Contract management

4.37 The ANZAC Systems Program Office Assurance Plan, requires the Systems Program Office to:

provide Governance and Assurance that outcomes intended from contracts relating to the FFH [ANZAC] Class are being achieved. Capturing the performance improvement opportunities reflected in these contractual arrangements, whilst providing assurance to Navy that a balanced, effective, seaworthy and affordable capability is available for Defence now and in the future, is critical to ensuring that the Strategic Objectives of the Commonwealth and Defence are achieved.72

4.38 There was evidence of active contract management and oversight by Defence’s ANZAC Systems Program Office including:

  • the establishment of an operational framework including engineering, material control, security and work health and safety within which the Alliance must operate;
  • monthly reporting from the Alliance to the ANZAC Systems Program Office on Alliance performance; and
  • review by the ANZAC Systems Program Office of Alliance engineering and supply chain acquisition proposals.

Appendices

Appendix 1 Department of Defence’s response to the audit

Appendix 2 Deployment of the Royal Australian Navy’s ANZAC class frigates

1. HMAS ANZAC is the first-of-class of the ANZAC class frigates. The ship deployed on: Operation Dirk (Border Protection) in 1997; the INTERFET peacekeeping taskforce to East Timor in 1999; Operation Damask in 2001; Operation Falconer (Iraq) in 2003; and Operation Slipper (Afghanistan) in 2002 to 2003, and 2012 to 2013. HMAS ANZAC completed an Anti-Ship Missile Defence upgrade in 2014, and commenced the ANZAC Mid-life Capability Assurance Program in September 2018 at Henderson Shipyard Western Australia.

2. HMAS Arunta deployed on: Operations Gaberdine (Immigration Support) and Relex (Coastwatch) in 2001; Operation Slipper in 2002; Operation Catalyst (Iraq) in 2007; and recently Operation Manitou (Middle East) in 2016 to 2017. Following HMAS ANZACHMAS Arunta completed the Anti-Ship Missile Defence upgrade in 2014. HMAS Arunta is the first of the frigates to undergo the ANZAC Mid-life Capability Assurance Program at Henderson Shipyard and is scheduled for completion in early 2018–19.

3. HMAS Warramunga deployed on: Operation Catalyst in 2006–2007; Operation Slipper in 2009; and Operation Manitou in 2017 to 2018. HMAS Warramunga completed the Anti-Ship Missile Defence upgrade in 2015.

4. HMAS Stuart deployed on: Operation Catalyst in 2004 and 2008; and Operation Slipper in 2009 to 2010 and 2011. HMAS Stuart completed the Anti-Ship Missile Defence upgrade in 2017.

5. HMAS Parramatta deployed on: Operation Catalyst in 2005 to 2006 and 2008; and Operation Slipper in 2010, and 2011 to 2012. HMAS Parramatta completed the Anti-Ship Missile Defence upgrade in 2016.

6. HMAS Ballarat deployed on: Operation Relex II (Border Protection) and Operation Catalyst in 2006; Operation Northern Trident (Round-the-world Expedition) in 2009; and Operation Manitou in 2018 and 2019. HMAS Ballaratcompleted the Anti-Ship Missile Defence upgrade in 2015.

7. HMAS Toowoomba deployed on: Operation Catalyst in 2007; Operation Slipper in 2009, 2011 and 2013; and Operation Manitou in 2014. HMAS Toowoomba completed the Anti-Ship Missile Defence upgrade in 2016.

8. HMAS Perth is the eighth and final ANZAC class frigate acquired by Navy. The ship deployed on Operation Manitou in 2016. HMAS Perth completed the Anti-Ship Missile Defence upgrade in 2010 being the trial ship for the major upgrade. HMAS Perth has been ‘laid-up’ at the Henderson shipyard since late 2017.

Appendix 3 ANZAC class frigates key performance measures, 2017–18

Navy’s Materiel Sustainment Agreement performance framework

1. Guidance on the development of the ANZAC frigates’ sustainment performance information is provided by Navy’s Materiel Sustainment Agreement Performance Framework.73 The Framework comprises a standard suite of:74

  • Six Key Performance Indicators—which provide guidance as to the effectiveness of actions and processes put in place between both parties to the agreement. There is a direct relationship between an individual Key Performance Indicator and a particular product outcome. The Framework requires performance measures to contribute directly to the achievement of one or all of the three product outputs of: materiel availability; materiel confidence; and sustainment efficiency.
  • Fourteen Key Performance Indicators—assist, through the identification and measurement of constraints, with the identification of contributing factors to future performance outcomes. Due to the nature of Key Health Indicators, they are best viewed in concert with other Key Performance Indicators and Key Health Indicators.
  • One strategic sustainment analytic indicator—high level health indicators used for cross product comparison of performance. They do not have an established target and do not require formal review and signoff.

2017–18 performance measures for the ANZAC class

2. Table A.1 provides an overview of the 2017–18 performance measures for the ANZAC class frigates, including the performance target.

Table A.1: 2017–18 performance measures for the ANZAC class

Performance measure Description Target
Key Performance Indicators
Monthly Materiel Ready Days achievement The number of Materiel Ready Days achieved compared to the number planned, expressed as a percentage. 100 per cent of materiel planned ready days.
External maintenance period milestone achievement The number of milestone failures occurring within a month, and the impact of those failures on successfully achieving the individual external maintenance period completion date. No stage milestone failures in an external maintenance period.
Year to date price achievement The percentage variance of actual year-to-date expenditure versus planned year-to-date expenditure. Zero variance with planned financial year to date price achievement.
Conformance to operating intent A measure of conformance to operation within the Statement of Operating Intent, with particular focus on operating profile, rate of effort and usage upkeep cycle. Zero exceptions by product in a calendar month.
Key Health Indicators
Systems Program Office staffing levels The number of Australian Public Service and military (funded) positions within the Systems Program Office at the end of each month which are filled by appropriately skilled and qualified staff. Expressed as a percentage. 90 per cent of positions filled by skilled and qualified staff.
Capability Manager Representative staffing levels The number of Australian Public Service and military (funded) positions within the Capability Manager Representative organisation at the end of each month which are filled by appropriately skilled and qualified staff. Expressed as a percentage. 90 per cent of positions filled by skilled and qualified staff.
Priority 1 materiel deficiency reports raised The number of Priority 1 materiel deficiency reports raised during the month and those that remain open at months end. Monthly Priority 1 materiel deficiency reports raised to be no greater than 10 per cent higher than the short-term rolling average of monthly Priority 1 materiel deficiency reports raised.
Open Priority 2 materiel deficiency reports The number of Priority 2 materiel deficiency reports raised during the month and those that remain open at months end. Open Priority 2 materiel deficiency reports at months end to be no greater than 10 per cent higher than the short-term rolling average Open Priority 2 materiel deficiency reports.
External maintenance period cost growth The percentage variance of final external maintenance period value versus the costed external maintenance period value. Less than 10 per cent cost growth in any external maintenance period event completing during the reporting period.
Organic level maintenance backlog The number of periodic organic level maintenance jobs overdue and inactive standard activities at the end of the month. Organic level maintenance jobs overdue at months end to be no greater than five per cent higher than the short-term rolling average of organic level maintenance jobs open.
External maintenance backlog The number of external maintenance jobs overdue and inactive standard activities at the end of the month. External maintenance jobs overdue at months end to be no greater than five per cent higher than the short-term rolling average of external maintenance jobs open.
External maintenance period effectiveness The average number of Priority 1 and Priority 2 materiel deficiency reports raised within 14 days of completion of all product external maintenance period activities completed within the month. The average number is not to be greater than 20 per cent higher than the short-term rolling average outcome.
Demand satisfaction rate The number of Government Furnish Equipment demands placed by platforms/stock owners for both inventory and Rotable spares which were delivered in full and on time. Expressed as a percentage. Demand satisfaction rate not more than five per cent less than the short-term rolling average demand satisfaction rate performance.
External maintenance period demand satisfaction rate Percentage of Government Furnish Equipment demands placed for external maintenance period activities for both inventory and Rotable spares. Not more than five per cent less than the short-term rolling average demand satisfaction rate performance.
NAVALLOWconfiguration effectiveness The number of NAVALLOW stock-codes issued that are recorded in the NAVALLOW, expressed as a percentage. 93 per cent NAVALLOW effectiveness.
Cannibalisation events The number of occurrences where cannibalisation was approved by Capability Manager Representative within the month. The number of monthly total product cannibalisation events is not more than 10 per cent higher than the short-term rolling average.
Open variations The number of open request for variations across the class at the end of the month. Open variations at months end to be no greater than five per cent higher than the short-term rolling average.
Open permanent engineering change proposals The number of open permanent engineering change proposals across the class. Open engineering change proposals at months end to be no greater than five per cent higher than the short-term rolling average.
Strategic Sustainment Analytic
CASG cost per Materiel Ready Day A rolling 12-month average cost per Materiel Ready Day for the platform No target.

Source: Defence.

3. Table A.2 provides an overview of the 2017–18 Key Performance Indicators traffic-light thresholds.

Table A.2: 2017–18 Key Performance Indicators traffic-light thresholds

Key Performance Indicator Green Amber Red
Monthly Materiel Ready Days achievement ≥ 95% ≥ 85% to < 95% < 85%
External Maintenance Period Milestone Achievement Zero stage failures in any active External Maintenance Period event. N/A One or more stage failures in any active External Maintenance Period event.
Year to date price achievement ≤ 3% variance of planned expense achieved. > 3% but ≤ 5% variance of planned expense achieved. > 5% variance of planned expense achieve.
Conformance to operating intent Zero exceptions > one exception N/A

Source: Defence.

4. Table A.3 provides an overview of the 2017–18 Key Performance Indicators traffic-light thresholds, as at June 2018.

Table A.3: 2017–18 Key Performance Indicators traffic-light thresholds, as at June 2018.

Key Health Indicator Green White Red
Systems Program Office staffing levels <-10% ≥-10% to ≤10% >10%
Capability Manager Representative staffing levels <-10% ≥-10% to ≤10% >10%
Priority 1 materiel deficiency reports raised <-10% ≥-10% to ≤10% >10%
Open Priority 2 materiel deficiency reports <-10% ≥-10% to ≤10% >10%
External maintenance period cost growth <-10% ≥-10% to ≤10% >10%
Organic level maintenance backlog <-5% ≥-5% to ≤5% >5%
External maintenance backlog <-5% ≥-5% to ≤5% >5%
External maintenance period effectiveness <-20% ≥-20% to ≤20% >20%
Demand satisfaction rate <-5% ≥-5% to ≤5% >5%
External maintenance period demand satisfaction rate <-5% ≥-5% to ≤5% >5%
NAVALLOW configuration effectiveness <-10% ≥-10% to ≤10% >10%
Cannibalisation events <-5% ≥-5% to ≤5% >5%
Open variations <-5% ≥-5% to ≤5% >5%
Open permanent engineering change proposals <-5% ≥-5% to ≤5% >5%

Source: Defence.

Appendix 4 Contractual sustainment arrangements with industry

Figure A.1: Evolution of ANZAC class frigates’ contractual sustainment arrangements with industrya

Note a: Tenix Defence was acquired by BAE Systems in 2008.

Source: ANAO analysis of Defence documentation.

Footnotes

1 Prime Minister, Minister for Defence Industry, Minister for Defence and Minister for Finance Joint Media Release: The Hunter Class — defending Australia and securing our shipbuilding sovereignty, 29 June 2018. https://www.minister.defence.gov.au/minister/christopher-pyne/media-releases/joint-media-release-prime-minister-minister-defence-2 [accessed 11 January 2019].

2 Parliamentary inquiries include: Joint Standing Committee on Foreign Affairs, Defence and Trade, Procurement procedures for Defence capital projects, August 2012, p. xxvii; Joint Standing Committee on Foreign Affairs, Defence and Trade, Review of the Defence Annual Report 2011–12, Canberra, June 2013, p. 90; Joint Committee of Public Accounts and Audit, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, Canberra, May 2014, and Report 448: Review of the 2013–14 Defence Materiel Organisation Major Projects Report, Canberra, May 2015, pp. 27–32.

3 ANAO comment: As noted in footnote 63, the effect of Defence’s comments on Recommendation no. 4 is to disagree to the recommendation.

4 The ANZAC class consists of 10 ships (all built in Williamstown) — eight operated by the Royal Australian Navy and two operated by the Royal New Zealand Navy. References to the ANZAC class frigates in this audit report refer to the eight Royal Australian Navy ANZAC class frigates. The Royal New Zealand Navy’s two ANZAC class frigates —HMNZS Te Kaha, commissioned by the Royal New Zealand Navy in July 1997, and HMNZS Te Mana, commissioned in December 1999 — will undergo a weapons and communications systems upgrade in 2018–19. The work will be undertaken in Canada by Lockheed Martin (Canada).

5 BAE Systems acquired Tenix Defence in 2008.

6 Prime Minister, Minister for Defence Industry, Minister for Defence and Minister for Finance Joint Media Release: The Hunter Class — defending Australia and securing our shipbuilding sovereignty, 29 June 2018. https://www.minister.defence.gov.au/minister/christopher-pyne/media-releases/joint-media-release-prime-minister-minister-defence-2 [accessed 11 January 2019].

7 The ‘Top 30’ are the 30 most costly Defence sustainment projects. More information can be found at http://www.defence.gov.au/Budget/18-19/2018-19_Defence_PBS_00_Complete.pdf [accessed 11 January 2019].

8 This is for baseline sustainment funding only — Defence does not include operational sustainment funding in Top 30 sustainment reporting for the ANZAC class. This is discussed further in Chapter 2 of this Report.

9 Until 30 June 2015, the sustainment provider was the Defence Materiel Organisation. In April 2015, the Government announced that it had accepted the recommendation of the 2015 Creating One Defence First Principles Review to delist the Defence Materiel Organisation as a separate entity and transfer its core responsibilities to CASG within Defence.

10 The Materiel Sustainment Agreement is discussed in Chapter 2.

11 Performance reporting is discussed in Chapter 3.

12 Systems Program Office, Defence’s Management of Materiel Sustainment Report 22017–18.

13 Parliamentary inquiries include: Joint Standing Committee on Foreign Affairs, Defence and Trade, Procurement procedures for Defence capital projects, August 2012, p. xxvii; Joint Standing Committee on Foreign Affairs, Defence and Trade, Review of the Defence Annual Report 2011–12, Canberra, June 2013, p. 90; Joint Committee of Public Accounts and Audit, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, Canberra, May 2014, and Report 448: Review of the 2013–14 Defence Materiel Organisation Major Projects Report, Canberra, May 2015, pp. 27–32.

14 The Auditor-General’s Report No.20 2018–19 Defence Major Projects Report 2017–18 provides coverage of this project.

15 Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements examined Defence’s administration of these agreements as at early 2015.

16 The other seven Materiel Sustainment Agreements are with Chief of Army, Chief of Air Force, Chief Information Officer, Chief of Joint Operations Command, Commander Joint Health Command, Deputy Secretary Strategy Executive, and Australian Geospatial – Intelligence Organisation.

17 The procedure was examined as part of Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements, paragraphs 2.35 to 2.44.

18 Navy operates aircraft, such as helicopters, which are subject to airworthiness regulations.

19 A Materiel Ready Day is any programmed day where a ship is not in an external maintenance period, not in extended readiness, or is not subject to a Priority 1 urgent defect report that because of its nature prevents the ship from achieving its current tasking.

20 Post-Rizzo refers to changes that occurred after the Plan to Reform Support Ship and Management Practices, (Rizzo Review) published in July 2011.

21 ANZAC sustainment Product Delivery Schedule.

22 Defence can receive additional funding from the Australian Government for the net additional cost of participating in major operations, where the net additional cost is equal to or exceeds $10 million in the financial year.

23 Navy’s Fleet Support Unit is made up of enlisted uniformed skilled technicians who maintain Navy’s ships and submarines. The Fleet Support Unit is a Navy resource and does not form part of the WAMA organisation or its funding arrangements. The Fleet Support Unit is funded through Navy’s operational budget.

24 The ANZAC class Alliance Management Office requested an extra $277 million to sustain the frigates in the period 2018–19 to 2024–25 as part of the WAMA proposal. This extra funding was not approved by Defence.

25 Fleet screenings are review activities discussed further in paragraphs 3.20 and 3.21.

26 Commonwealth of Australia, Plan to Reform Support Ship and Management Practices, (Rizzo Review), July 2011, p. 35.

27 See Footnote 19 for a definition of Materiel Ready Day.

28 Defence has also authorised a Statement of Operating Intent for the ANZAC class frigates, which specifies the frigates’ configuration, role and operating environment. This includes a description of: the intended and approved tasks; roles; functions; capability; operating limitations; operating profiles; operating environment; the rate of effort; and the Usage Upkeep Cycle.

29 Seaworthiness is the platform and its systems are operated and supported in accordance with a capability manager’s authorised operating and support intent, such that the likelihood of achieving a specified operational effect for a defined tasking is maximised and hazards and risks to personnel, the public and the environment have been eliminated or minimised.

30 United Kingdom National Audit Office, Investigation into equipment cannibalisation in the Royal Navy, 1 November 2017, p.5. Available here: https://www.nao.org.uk/wp-content/uploads/2017/10/Investigation-into-equipment-cannibalisation-in-the-Royal-Navy.pdf [accessed 11 January 2019].

31 A Gate Review is an internal assurance process intended to supplement regular management processes.

32 Warship Asset Management Agreement, FFH to Future Frigate Capability Transition De-Risking Study: Stage 1 Final Report, p.23.

33 Defence internal advice, November 2015.

34 ‘Parent navy’ relates to the Royal Australian Navy operating a ship class with unique design and/or sub-systems.

35 Commonwealth of Australia, Plan to Reform Support Ship and Management Practices, (Rizzo Review), July 2011, pp. 23–24.

36 Advice to the Deputy Chief of Navy December 2015.

37 Priority 1 and 2 Materiel Deficiency reports are the same as Priority 1 and 2 URDEFs.

38 The Plan is discussed further in Auditor-General Report No.39 2017–18Naval Construction Programs — Mobilisation.

39 Department of Defence, 2016 Integrated Investment Program, p. 80.

40 Australian Strategic Policy Institute (2018) Thinking through submarine transition, pp. 8-9.

41 Auditor-General Report No.39 2017–18 Naval Construction Programs — Mobilisation, p. 11.

42 The difference between a Key Health Indicator and Key Performance Indicator is outlined in Appendix 3. A strategic sustainment analytic indicator is a high-level health indicator used for cross product comparison of performance. There is no established target and formal review and signoff of reported information is not required.

43 Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment, paragraph 3.7.

44 Auditor-General Report No.30 of 2014–15 Materiel Sustainment Agreements, p. 106.

45 As noted in paragraph 3.3 this was previously a Key Performance Indicator.

46 Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment, paragraph 3.8.

47 See paragraphs 4.31 and 4.32.

48 It is a web-based system designed to provide performance reports to CASG and Capability Managers. Data is entered monthly, usually by personnel in the relevant Systems Program Office. The use of the system was examined in Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment.

49 As noted earlier in this chapter, the key performance measures are relevant and partly reliable, but do not provide a complete picture of sustainment performance, primarily because the indicators do not address sustainment outcomes and efficiency.

50 In October 2017, HMAS Perth (FFH-157) was scheduled to be reactivated following the completion of scheduled maintenance; however, it remains in lay-up due to crew shortages (as shown in Figure 3.1 and Figure 3.2). Navy has advised a crew is expected to become available between July 2019 (medium confidence) and January 2020 (high confidence).

51 A Product of Interest is a sustainment product that requires increased management attention and support as there have been variances in the availability and/or cost performance.

52 Defence advised cross levelling is the authority and ability to shift resources, including funds, from one product to meet the requirements of another. At the theatre strategic level and operational level, it is the process of diverting resources from one Major Surface Ship Product to meet the higher priority of another within the Director General Major Surface Ship’s directive authority.

53 Auditor-General Report No.2 2017–18 Defence’s Management of Materiel Sustainment (paragraphs 3.57-62) reviewed the public reporting of sustainment activity and found there were opportunities to increase the completeness and transparency of publicly reported information regarding materiel sustainment.

54 Product Delivery Schedule of the ANZAC Materiel Sustainment Agreement.

55 ISO 55000 Asset management—overview, principles and terminology, and its related standards: ISO 55001 Asset management—management systems requirements, and ISO 55002 Asset management—management systems—guidelines for the application of ISO 55001 provide a framework for the management of an asset. See ISO 55000, Asset Management—overview, principles and terminology, 2014, p. 1.

56 Commonwealth of Australia, Plan to Reform Support Ship Repair and Management Practices (the Rizzo Review), July 2011, p. 29.

57 Commonwealth of Australia (2015), First Principles Review Creating One Defence, p. 36.

58 Division 2 of the Commonwealth Procurement Rules sets out the conditions that allow a limited tender to be conducted.

59 Section 4.4 of the July 2014 Commonwealth Procurement Rules, which applied at the time, stated that:

Achieving value-for-money is the core rule of the CPRs. Officials responsible for a procurement must be satisfied, after reasonable enquires, that the procurement achieves a value-for-money outcome. Procurements should:

a. encourage competition and be non-discriminatory;

b. use public resources in an efficient, effective, economical and ethical manner that is not inconsistent with the policies of the Commonwealth;

c. facilitate accountable and transparent decision making;

d. encourage appropriate engagement with risk; and

e. be commensurate with the scale and scope of the business requirement.

60 The Defence reviewers comprised Commonwealth personnel from different areas of expertise including commercial, technical, legal and financial.

61 The ANAO’s review of the publicly available Cabinet Minute of 14 August 1989 relating to the ANZAC Ship Projectindicates that Cabinet did not consider sustainment costs at that time. National Archives of Australia https://recordsearch.naa.gov.au/SearchNRetrieve/Interface/ViewImage.aspx?B=31430880 [accessed 11 January 2019].

62 These challenges were discussed in Chapter 2 of this Report.

63 ANAO comment: Resource Management Guides and the Budget Process Operational Rules are issued by the Department of Finance. The effect of Defence’s comments is to disagree with the recommendation, as Defence does not issue this guidance and its response does not recognise the gap in its own guidance. As discussed at paragraphs 4.14 – 4.19 of this audit report, there was uncertainty as to whether ANZAC sustainment funding aligned with original Government approvals. Defence did not seek advice from central agencies to clarify whether it had the necessary funding approvals and whether it should raise this matter with the Minister for Finance. Recommendation 4 is that Defence clarify its processes for seeking approvals in such circumstances.

64 Navy’s Fleet Support Unit is made up of enlisted uniformed skilled technicians who maintain Navy’s ships and submarines. The Fleet Support Unit is a Navy resource and does not form part of the WAMA organisation or its funding arrangements. The Fleet Support Unit is funded through Navy’s operational budget.

65 Auditor-General Report No.17 2014–15 Recruitment and Retention of Specialist Skills for Navy, p. 96.

66 Under the WAMA, Key Performance Indicators are the only performance measures that are payment related.

67 This is equivalent to the internal Key Health Indicator ‘priority 1 Materiel deficiency reports raised’.

68 This Key Performance Indicator was not assessed ‘by mutual agreement between the Commonwealth and Industry Participants until Program Agreement 2’ as the sustainment arrangements were still in transition.

69 One Key Health Indicator — ‘funding adequacy over forward estimates’ — was not measured as metrics were not established through the Transition Period.

70 The Strategic Performance Measures assessed were: Safety Culture; Materiel Seaworthiness Status; Relationships; Best For Program/Alliance; and Continuous Improvement and Efficiency Performance.

71 As discussed earlier in paragraphs 4.11 to 4.15.

72 Department of Defence, ANZAC Systems Program Office CP, 003 Assurance Plan, February 2018, p. 6.

73 Auditor-General Report No.30 2014–15 Materiel Sustainment Agreements discussed and assessed Navy’s Materiel Sustainment Agreement Performance Framework, see paragraphs 5.11- 5.17. The report also assessed the Key Performance Indicators then reported on for the ANZAC frigates, see paragraphs 5.18-5.20.

74 The framework is designed to be scalable and acknowledges that not all measures will be applicable to all products. The six Key Performance Indicators are: monthly Materiel Ready Days achievement; service level achievement; rate of effort/aircraft availability achievement; achievement of external maintenance period planning milestones; conformance to operating intent; price reliability. The Key Health Indicators are those specified in Table 3.1.