Steve Taitoko

Dr Tomas Remenyi

Sacha McMonagle-Ihasz

As you probably already know, we are solving a complex problem. Around the country, hundreds of scientists produce petabytes (PB) of the most up-to-date data about the world’s climate. Our scientists and researchers are creating a wealth of high-quality, high-resolution climate model outputs. This data can help researchers, government, industry, or the general public make better decisions about Australia’s future.

The problem is that the quantity of information they produce is hard to share between researchers or other people who need it. Until now, the supply of this information has not been keeping up with its growing demand.



The solution is the Federated Climate Data Initiative (FCDI).



We are creating a single, united virtual collection from the enormous amounts of information currently stored around the country. The aim is to make the data much easier to find, access and use, dramatically enhancing the quality of future research and the services built this new knowledge, particularly around future risk.



Australia’s many regional climate projections are vital for understanding what impacts we must adapt to, for example, an increased bushfire risk. We can then determine which risks we can reduce, which could involve changing how we do things, such as what we grow and where we grow it. And, importantly, we can work out which impacts we want to avoid, for example, losing precious ecosystems if we allow climate change to increase by 2°C. By accelerating how we create products and services from these projections, change-makers will be able to act on the findings sooner.

Although the Federated Climate Data Initiative has been in the planning since 2018, as of April 2021, the Department of Industry, Science, Energy and Resources Disaster Management and Risk Reduction Fund has officially funded it with initial seed funding of $500,000. Development of contractual arrangements with partners is ongoing.

Since April, we have been busy setting up data access nodes across the country. We have installed nodes in Tasmania, Western Australia, Queensland and Victoria. They are steadily adding to the data available through the network.

We are building nodes in South Australia and within the National Computing Infrastructure are well underway. We are also working with the Australian Climate Service, the Climate Services for Agriculture Program and the National Environmental Science Program on how the FCDI can support them best. We will keep you updated on these developments through future newsletters.

All state and territory governments have now joined the project. The CSIRO, the Bureau of Meteorology and the Department of Home Affairs are also contributors. We’ve also welcomed some of Australia’s leading universities to the team including:

And welcoming our new implementation partner, Eratos, who is helping the project come to life.

If you’d like to part of this historic project, we would love to hear from you. To get involved or if you want to promote complimentary activities through our newsletter, please contact the Chief Investigator, 

In each issue of the newsletter, we will put the spotlight on each team contributing to this important collaboration.

The first spotlight is on the newest player in this field, Eratos, with CEO Steve Taitoko Eratos is one of the key players in the FCDI. The Melbourne based startup is responsible for building the infrastructure that the FCDI is based on. Tom Remenyi met with Steve Taitoko for the first time about two years ago, and they immediately knew they needed to work together to make the project happen.

What does Eratos hope to achieve through the FCDI?

Steve describes Eratos as a company “that’s looking to change how we make decisions about the world we live in.” They do that by creating the technology to allow people to make decisions from data that hasn’t been available before, data based on real-time information about the world we live in.



“Our purpose is how we can help humans survive on this ever increasingly heating rock that’s hurtling through space,” Steve says. “We can use a lot more information to create shared and collaborative knowledge to make better decisions about the world we live in. Decisions like what we plant in our soils, how we build our cities, how we can ensure that humans and our ecosystems survive within a changing climate, how we adapt to more extreme climate events that are going to be happening.”

Steve says that the Federated Climate Data Initiative and Eratos fit together like two pieces of a puzzle. “I was going to Hobart for a couple of days holiday, and I organised some meetings while I was down there. One of those meetings happened to be with Dr Tom Remenyi.”



“He showed me a diagram of an ARC linkage proposal… and I said to him, that diagram you’ve just drawn, that’s what we’re building, so don’t worry about that part; we will take care of that.” And through that meeting, a serendipitous partnership was born. Tom would focus on climate science, and Eratos would support him with the platform build. “That’s how it all started,” Steve says. “It was really random, just a random meeting.”



To hear more from Steve Taitoko and other key players in the project; keep an eye out for our new podcast. We will keep you updated on that. 

We are solving a complex problem. Our scientists and researchers are creating a wealth of high-quality, high-resolution climate model outputs. This data can help researchers, government, industry, or the general public make better decisions about the future.

Cover

Federated Climate Data Initiative Newsletter

Throughout human history there have been significant innovations that have defined our civilisation in the way we think and act. For several million years our actions and thoughts were defined in the paradigm of Hunting and Gathering. Following this period through multigenerational innovations we entered the Agricultural Age which extended for thousands of years. Subsequent inventions of the printing press and reading glasses generated a foundation for our collective civilisational consciousness to expand. This gave way to exponential innovations that allowed us to arrive more rapidly into the Industrial Age and Information Age.

Today, we're at the precipice of our next significant era - the “Augmented Age". History indicates that this exponential innovation horizon will usher a new age resulting in more rapid and radical changes in the way we solve problems - further accelerating the expansion of our collective consciousness and innovations.

The previous asymmetry between our digital and physical worlds is changing, impacting our shared reality. We are begging to see the coupling of our digital world and physical worlds. Manufacturing and utilities are measuring vital performance increases using the Internet of Things (IoT). Surgeons are experimenting with holographic anatomic models during surgical procedures. Governments are generating digital twins of their cities as virtual test environments to help improve decision-making and scenario-planning.

Still, today's technological applications are just snippets of what could be accomplished. To accelerate and maximise our potential we must design systems in which humans and machines function collaboratively in ways that complement each other's strengths and counterbalance each other's limitations. Our inherent human abilities are going to be augmented by computational systems transporting us into the Augmented Age.

"Men will set the goals, formulate the hypothesis, determine the criteria, and perform the evaluations. Computing machines will do the routinizable work that must be done to prepare the way for insights and decisions in technical and scientific thinking. . . . The symbiotic partnership will perform intellectual operations much more effectively than man alone can perform them."  

To expand human and computer intelligence, we must explore the relationship by understanding and transforming our perspective on how we interact with technology:


We must shift perspectives from seeing machines as separate entities that fundamentally are technologies that automate tasks, to exploring the idea that machines' capabilities will be used 

 expanding our collective consciousness and improving the abilities of humans,

Consider the full potential of both humans and machines evolving mutually creating more 

rather than the approach of machines as a tool for reducing 

We need to develop a more general outlook of 

 for human-computer collaboration, rather than a human-dictated view of deploying technologies.

We need to change our focus to a more cohesive cyber-physical world rather than individual physical or virtual worlds.

Through changing our perspective towards working together and augmenting each other’s work, we will start to occupy a new space of advancement and produce unique ideas and solve problems in new and beautiful ways.

What is this new space, and how will this human and machine collaboration work?

Exploring this question is a project called 

 conducted by Autodesk. The project’s objective was a prototype partnership of algorithms, robots and humans, working collectively to solve a remarkably complicated design problem; the task was to build a pavilion.

The humans navigated around the construction site, shaping and forming the elements. They carried out the labourers tasks, which is very difficult for robots to accomplish because of their non-isomorphic body. Conversely, the machines did the fibre winding, which is virtually impossible for a human to do. And all of this being directed by an AI system, telling the humans and robots what to do and managing and tracking the numerous individual components. Most excitingly the construction of this pavilion was not possible without humans, machines and algorithms collaborating and augmenting each other’s work.

Humans, machines and a better process for operating algorithms will side-by-side yield more reliable outcomes than both the most gifted humans or the most advanced algorithms working in isolation. The need to design these systems for human-computer collaboration demands more attention.

The late Steve Jobs explained augmentation and the concept of human-machine collaboration very well.

"I think one of the things that really separates us from the high primates is that we're tool builders. I read a study that measured the efficiency of locomotion for various species on the planet. . . . Humans came in with a rather unimpressive showing, about a third of the way down the list, but somebody at Scientific American had the insight to test the efficiency of locomotion for a man on a bicycle. . . . A human on a bicycle blew the condor away, completely off the top of the charts. And that's what a computer is to me . . . it's the most remarkable tool that we've ever come up with; it's the equivalent of a bicycle for our minds." 

Steve Jobs is recognised for introducing human-centric design thinking into the personal computer. For us to achieve human-machine symbiosis and the "Augmented Age" it will require a similar foundation of design thinking into the domains of, robotics, data science and AI.

The state of things to come will be unlike anything we've ever seen before. We're going to have a world with more variation, more connectedness, more dynamism, and, of course, more beauty. Why? Because we will be redesigning the relationship between technology, nature and humanity.

Today, we're at the precipice of our next significant era - the “Augmented Age". History indicates that this exponential innovation horizon will usher a new age resulting in more rapid and radical changes in the way we solve problems.

Cover Image

Lesh, Neal & Marks, Joe & Rich, Charles & Sidner, Candace. (2004). "Man-Computer Symbiosis" Revisited: Achieving Natural Communication and Collaboration with Computers.. IEICE Transactions. 87-D. 1290-1298.

J. C. R. Licklider, “Man-computer symbiosis,” IRE Transactions on Human Factors in Electronics, March 1960, http://worrydream.com/refs/Licklider%20 -%20Man-Computer%20Symbiosis.pdf. In this essay, Licklider analogized the relationship of humans and computers with that of the fig wasp and the fig tree.

Jobs made this comment in the 1990 documentary film Memory & Imagination: New Pathways to the Library of Congress. Clip available at “Steve Jobs, ‘Computers are like a bicycle for our minds.’ - Michael Lawrence Films,” YouTube video, 1:39, posted by Michael Lawrence, June 1, 2006, https:// www.youtube.com/watch?v=ob_GX50Za6c.

References

Design For The Augmented Age

People regularly ask what the meaning of Eratos is. In short, we named the company after 

, the Greek philosopher, mathematician, and scholar who lived around 200 BC. Among his many important achievements, he was the chief librarian at the Great Library of Alexandria, and it was at that institution that the earliest recorded examples of metadata can be found.

Metadata simply put, is data about data. It was applied to scrolls at the library by attaching small tags containing the title, subject and author, allowing library users to assume the content, without having to unroll each scroll and so they could be returned to their rightful location after use. Metadata today still apply that simple principle, but with a little more detail.

The use of metadata is not just a convenient ‘nice to have’, it is a ‘must-have’, particularly as we design more advanced analytics relying upon machine-to-machine communications. Mixing structured and unstructured data into automated analytic processes, mandates rapid access to data about the data, to retrieve precise, relevant and quality data.

, we discussed the power of the FAIR principles and how they are foundational in the design of Eratos. The key to achieving a data system based on FAIR principles is robust metadata. In this context:

Metadata makes data findable by allowing search on the elements of metadata that tell the story of the data.

Metadata makes data accessible by creating elements within metadata that provide access control and permissions management.

Metadata makes data interoperable through the standardisation of the metadata’s vocabularies and formats.

Metadata makes data reusable by providing rich contextual provenance about the data’s origins, use, licence, and governance.

In making data findable and accessible, the task is relatively straight forward, relying on the addition of descriptive elements to metadata. However, making data interoperable and reusable requires a high level of coordination to ensure metadata is created consistently, regardless of how it was generated or by who. To ensure standardisation, frameworks called schemas are required.

A schema provides the structure of a metadata collection to a specific domain, such as published research. They establish a set of rules that define required information, describing the semantics and relationships of the elements, for example 

 (DOIs) which are persistent interoperable identifiers for use on digital networks, adopted by scholars as a means to describe published research.

Much effort has been made to create standard schemas through international validation by organisations such as the International Standards Organisation. One of the most commonly used standards is the 

 (DCMI). Used often on webpages, it is simple and generic with just 15 elements, containing information such as title, date, type, format, creator, and rights.

Eratos utilises standard schemas to ensure metadata across all fields of cyber-physical data is interoperable and reusable. As they are machine-readable, schemas provide scalability, particularly as we move to more data generated and stored at the edge and are a vital component as we build the world’s semantic web of cyber-physical data.

Without standardised schemas and metadata, it will be impossible to extract needles from a rapidly expanding data haystack at speed and scale. Thankfully, the groundwork laid by our academic community allows us to bring robust and uniform metadata into the mainstream and for that, we have better data.

Metadata simply put, is data about data. The use of metadata is not just a convenient ‘nice to have’, it is a ‘must-have’, particularly as we design more advanced analytics relying upon machine-to-machine communications.

Better Data With Metadata

Data is generated and captured in every aspect of our daily lives these days. As individuals, the phones in our pockets, watches on our wrists, our televisions, and even the cars we drive. On the industrial scale, we have the burgeoning Internet of Things (IoT), with myriad sensors deployed on everything from 

This data is being generated from what is broadly called, cyber-physical systems (CPS), the intersection of the physical world with digital devices, sensors, computation, control and networking.

Like the internet connected people to information, these CPS are connecting information to the physical world. CPS are what underpins our transition to the 

Founding executive editor, Kevin Kelly wrote in 

 of a world where “someday soon, every place and thing in the real world—every street, lamppost, building, and room—will have its full-size digital twin in the mirrorworld.”

This mirrorworld “will reflect not just what something looks like but its context, meaning, and function. We will interact with it, manipulate it, and experience it like we do the real world.”

Technologies such as digital twins will form the basis of the mirrorworld, creating a living 3D metaverse that mimic the real-world environment. Loaded with sensors streaming data from everywhere, these digital avatars will provide infinite analytical possibilities, allowing humans and machines to diagnose, optimise, and predict in ways never before dreamt of.

Much of the data generated by CPS has never existed before, providing many exciting opportunities to gain fresh insights about our natural and built worlds. However, transforming data at the scale required comes with many challenges.

Of most concern is our inability to cope with the data flowing from CPS. A typical data scientist will lay awake at night worried about the three V’s: Volume (amount of data); Velocity (the speed at which data is generated); and, Variability (differences in data types, quality, programming languages). With the onslaught of data coming down the pipe from CPS, the size of problems caused by the three V’s are beyond most data scientists’ capability to control.

It seems incongruent to say that the existence of CPS is the culmination of centuries worth of research and technological innovation, yet they appear to have snuck up on us, given our unpreparedness to cope with the data explosion they have caused .

Fortunately, there are many researchers, data scientists, and organisations collaborating to harness CPS to generate quantum leaps in the way in which we interact with our natural and built worlds.

Take for example the groundbreaking work undertaken by Australia’s preeminent research organisation the CSIRO. Within their cyber-physical systems unit, they are 

 to better understand what factors are affecting colonies and pollination to reverse the unprecedented decline in bee populations globally.

One of the largest scale CPS programs in the world is 

. Along with digitising all government services, the program is focussed on creating interconnected systems for urban living, mobility, and health. For example, town planning is linked to transport design to plan for the seamless integration of autonomous vehicles into their society.

We believe in the potential that CPS has to positively reshape our world, and are committed to solving the many data problems inherent in translating raw sensor data into meaningful, actionable insights about our natural and built worlds.

The idea of a mirrorworld is interesting, but right now, we’re not quite sure what’s looking back at us. If we have anything to do with it, that’s about to change.

Like the internet connected people to information, Cyber-physical systems are connecting information to the physical world. They underpin our transition to the 4th Industrial Revolution.

The Rise of Cyber-Physical Systems

There exists a well-worn phrase that ‘data is the next oil’, the irony of which appears lost given the technologically-driven shift to renewable energy. However, as the inventor of the internet, Tim Berners-Lee said, “data is a precious thing and will last longer than the systems themselves”, suggesting that data has more than a single useful life.

The idea of data as reusable or temporal in nature isn’t a new concept, however, even amongst the research community, there is a growing realisation that outdated systems and lack of data governance, have resulted in significant amounts of data locked-up inside institutions. The result is some of our most highly prized data, created by some of our brightest minds, are gathering virtual dust at the bottom of digital filing cabinets.

In recognition of this problem, in 2016 a group of scientists developed the 

 to establish guidelines that support the reusability of scholarly data, making data findable, accessible, interoperable, and reusable. The catalyst for this initiative was driven by artificial intelligence’s insatiable appetite for data with minimal human intervention, or what the group called 

Clearly, data practices of the past were standing in the way of progress for the machine-to-machine future. So important is the need to create more accessible and reusable data, that at the 2016 G20 Hangzhou Summit, G20 leaders 

 endorsing the application of FAIR principles as a vital component of science and technology innovation-driven growth.

 in Data Intelligence, researchers set out how FAIR principles could be implemented to achieve machine-actionability. It noted under each principle that:

Data should be easy to find for both humans and computers, where machine-actionable metadata are essential for process automation.

 Access control is essential for human and machine interaction with data, with strong governance.

Resources attributed to data should be structured so that machines can aggregate them into a richer, unified view of the data and its resources. To enable this, the semantics of each resource must be clearly defined.

 Descriptions of data through its metadata and resources are sufficient for humans and machines to decide whether the data is relevant for reuse, and if so, under what conditions should it be used.

This matters because we have an arms race of sorts, to integrate artificial intelligent systems into all parts of our lives. A prerequisite for this to happen is that data must be AI-ready.

The US government’s biomedical and health research organisation, the National Institutes of Health (NIH) are moving into 

, and have adopted FAIR principles to help guide their transition. One of the core themes of their plan, AI-ready data, means that the “creation and stewardship of data sets to enable machine learning may be the NIH’s single greatest lever to accelerate the application of AI within biomedicine.”

It is no surprise that organisations like the NIH are looking to leverage artificial intelligence to transform their operations. McKinsey & Company 

 to add US$5.8 trillion in global value annually by 2035. They estimate that only 20% of AI-aware companies are using the technology in a core business process or at scale. Four out of the five challenges causing this low adoption relate to data that is not AI-ready.

It is for this reason, the FAIR principles guide the development of Eratos. In applying these to cyber-physical systems, there is a clear path toward unlocking the vast treasure trove of data that already exists about our natural and built worlds. With the right framework, this also provides a significant opportunity to new data streams, given they will plug into robust data infrastructure, leveraging integrated systems and automation, to continually build deep, rich, contextual data.

Eratos is taking the tracks laid by academia and building a bridge to apply it to similar problems in the commercial world. Using the FAIR data principles provides an elegant and considered solution, to address a broad and complex problem.

A group of scientists developed the FAIR data principles to establish guidelines that support the reusability of scholarly data, making data findable, accessible, interoperable, and reusable.

Making Data Fair

As a term, ‘data science’ is a dominant feature within our information systems lexicon. It is referred to often, but do we truly understand what it is and how best to unleash its power as a force for good?

The concept was first used in the early 1970s, however, data science as a term wasn’t codified until 

’s 2001 paper.  Since then, it has been interchangeably used in emerging areas such as big data, artificial intelligence, and IoT, with little regard to how to apply and extract the greatest value from it. As a result, significant barriers exist to the mainstream adoption and scalability of data science.

At its core, data science encompasses data strategy, engineering, analytics and visualisation. All of this must then be wrapped within the context of particular domain expertise to be useful. Our typical response has been to wrap responsibility for all of this into one tidy role description.

Finding one, let alone the hundreds of thousands of individuals required who have the skill, experience, and wherewithal to meet the expectations of that job description, is like finding the proverbial unicorn. Welcome to the world of the data scientist, and thus the pinch point of our data science supply chain.

Kaggle State of Data Science and Machine Learning

 survey of over 16,000 data professionals, a number of recurring themes were identified as challenges faced by data scientists, including data curation, management support and budget approval.

Given the vast scope of responsibility we are asking our data scientists to take on and the structural barriers in their way, we have a mountain to climb to reach the full potential of advanced analytics.

To resolve the issue of the vast scope of a data scientist role, we must look at ways to treat data science as a collaborative workflow and not a single data scientist role. Breaking out the core functions of strategy, engineering, analytics and visualisation into distinct roles, will be a good start to build cross-functional teams with domain expertise in each function.

To resolve the second challenge of breaking down barriers to better data management, we must address some of the following core issues:

It is estimated that a data scientist spends 

80% of their time finding and cleaning suitable data

 before they can even use it. This creates an even wider gap between the application of their domain expertise and the scale of problems they are solving.

The way in which we are interacting with data is changing rapidly. For instance, it is estimated that 

75% of data will be generated and processed on the edge by 2035

, a big shift from the current 10%. Combine this with 

, and we have constructed the perfect barriers to full data flow and collaboration.

Data privacy, sovereignty, and shareability are significant barriers to access and collaboration. Without coherent governance and policy management built into data management tools, interoperability within and external to organisations is almost impossible.

Often referred to as data velocity, incoming data is growing exponentially. It is estimated that we currently have over 

. To put that in context, if you took an average smartphone’s storage capacity, 44ZB would fill 188 trillion phones, which if laid end-to-end would circumnavigate the earth about 670 times.

. A threefold increase in data storage at the scale of data that already exists presents a big problem given the way in which our data is currently architectured and governed.

These hardwired and structural barriers have resulted in the burden of ingesting and processing data at scale falling on the few. To resolve this, we must shift ownership to the highest levels of our institutions. This will involve: redesigning our systems; building strong governance settings; establishing data teams; creating better tools; and, providing pathways to training.

Ultimately, to truly become AI-ready we must build a data culture capable of allowing science to flourish, unlocking new value, and solving the wicked problems faced by humanity.

Scientific advances, real world benefits

Journal entries