Schema Evolution and Schema Biographies: Studying the Lives of Schemata

Our research on schema lives studies how entire relational database schemata evolve over time. Empirical studies of up to 195 open-source software projects show that schema evolution ranges from almost complete stability to highly active change, with much evolution concentrated in short periods of a project's lifetime. We have identified recurring evolutionary taxa and timing patterns, while earlier studies using Lehman's laws showed that schemata typically grow slowly and exhibit long periods of calmness interrupted by focused maintenance activity.

Large-Scale Patterns and Taxa of Schema Evolution

In a paper in ICDE 2021, we present the findings of a large study of the evolution of the schema of 195 Free Open Source Software projects. We identify families of evolutionary behaviors, or taxa, in FOSS projects. A major finding has been that the absence of schema evolution is more prevalent than its presence: a large percentage of the projects demonstrate very few, if any, actions of schema evolution. Two other taxa involve the evolution via focused actions, with either a single focused maintenance action, or a large percentage of evolution activity grouped in no more than a couple interventions. Schema evolution also involves moderate, and active evolution, with very different volumes of updates to the schema. To the best of our knowledge, this is the first study of this kind in the area of schema evolution, both in terms of presenting profiles of how schemata evolve, and, in terms of the dataset magnitude and the generalizability of the findings.

In a follow-up paper in Information Systems, 2022, we expand the aforementioned contributions. We investigate how the different taxa relate to measurable properties of schema evolution, specifically, duration of schema and project updates, activity volume, and heartbeat. We show that although different taxa have practically very similar duration, the evolutionary characteristics differ in analogy to the "active" character of each taxon. Moreover, by observing certain similarities in the measurable properties of the taxa, we take the opportunity to introduce super taxa, which complement the previous taxonomy with the groupings of the aforementioned taxa in terms of overall profile similarity, resulting in a more concise and intuitive taxonomy, providing a cleaner separation of evolution measures. Finally, we show that schema evolution is frequently, a time-concentrated activity.

A third paper in EDBT 2025, discusses the problem of patterns of how schemata evolve over time. To come up with these timing patterns, we have studied the 151 projects of our dataset of schema histories that came with duration more than 12 months. The eight timing patterns of schema evolution are largely grouped in 3 families: (a) the family of limited and focused-in-time change, whose members differ only on when the change happens (early, middle or late life of a project), (b) the family of regular updates, whose members differ in the frequency of change, and (c) the family of middle or late initiation of change, whose members differ in the initiation of schema maintenance as well as the frequency of change.

Panos Vassiliadis, Alexandros Karakasidis. Time-Related Patterns of Schema Evolution. 28th International Conference on Extending Database Technology (EDBT 2025), pp: 310-323, Barcelona, Spain, March 25-28, 2025

Focus: Identifies eight recurring timing patterns of schema evolution in 151 long-lived projects, organized into three broad families of focused, regular, and delayed change.
DOI: 10.48786/edbt.2025.25.
Related material: local page for Time-Related Patterns of Schema Evolution with paper and supplementary material; EDBT 2025 proceedings at OpenProceedings.

Panos Vassiliadis, George Kalampokis. Taxa and super taxa of schema evolution and their relationship to activity, heartbeat and duration. Information Systems, Volume 110, 2022, 102109, ISSN 0306-4379, doi:10.1016/j.is.2022.102109.

Focus: Relates schema-evolution taxa and super-taxa to activity, heartbeat, and duration, showing that evolution is often concentrated in limited periods of project life.
DOI: 10.1016/j.is.2022.102109.
Related material: local page for Taxa and super taxa of schema evolution with paper and supplementary material.

Panos Vassiliadis. Profiles of Schema Evolution in Free Open Source Software Projects.37th IEEE International Conference on Data Engineering (ICDE '21), Chania, Crete, Greece, 19-22 April 2021.

Focus: Classifies the schema histories of 195 FOSS projects into recurring evolutionary profiles, with very limited or absent schema change proving more common than highly active evolution.
DOI: 10.1109/ICDE51399.2021.00008.
Related material: local page for Profiles of Schema Evolution in Free Open Source Software Projects with paper, presentation, data, code, and results.

Schema Evolution and Lehman's Laws

In this line of research, we have performed a thorough study on the evolution of databases that are part of larger open source projects, publicly available through open source repositories. Our first attempt towards unveiling how schemata evolve involved using Lehman's laws of software evolution, a well-established set of observations on how the typical software systems evolve (matured during the last forty years), as our guide towards providing insights on the mechanisms that govern schema evolution.

Our findings indicate that the schemata of open source databases expand over time, with long periods of calmness connected via bursts of maintenance effort focused in time, and with significant effort towards the perfective maintenance of the schema that appears to result in an unexpected lack of complexity increase. At the same time, unlike typical software systems, the incremental growth of the schema is typically low and its volume follows a Zipfian distribution. Still, although the technical assessment of Lehman's laws shows that the typical software systems evolve quite differently than database schemata, the essence of the laws is preserved: evolution is not about uncontrolled expansion; on the contrary, there appears to be a stabilization mechanism that employs perfective maintenance to control the otherwise growing trend of increase in the information capacity of the database.

Ioannis Skoulis, Panos Vassiliadis, Apostolos V. Zarras. Growing up with stability: How open-source relational databases evolve. Information Systems, Volume 53, October - November 2015, Pages 363 - 385. doi:10.1016/j.is.2015.03.009.

Focus: Applies Lehman's laws to relational schema histories and finds slow growth, long calm periods punctuated by maintenance bursts, and stabilization through perfective maintenance rather than uncontrolled complexity growth.
DOI: 10.1016/j.is.2015.03.009.
Related material: local page for Growing up with stability with highlights, papers, presentations, data, code, and results.

Ioannis Skoulis, Panos Vassiliadis, Apostolos Zarras. Open-Source Databases: Within, Outside, or Beyond Lehman's Laws of Software Evolution?. 26th International Conference on Advanced Information Systems Engineering (CAiSE 2014). 16-20 June 2014, Thessaloniki, Hellas.

Focus: Provides the initial macroscopic assessment of relational schema evolution through Lehman's laws, showing that database schemata evolve differently from typical software while still exhibiting stabilizing evolutionary mechanisms.
DOI: 10.1007/978-3-319-07881-6_26.
Related material: local page for Open-Source Databases: Within, Outside, or Beyond Lehman's Laws of Software Evolution? with highlights, paper, presentation, data, code, and results.

Keynotes, Talks, and Videos

Panos Vassiliadis. Schema Evolution and Gravitation to Rigidity: a tale of calmness in the lives of structured data. 7th International Conference on Model and Data Engineering (MEDI 2017), October 4-6, 2017 - Barcelona.

[A paper from a keynote talk at MEDI 2017. Plz., visit the Talks page for the talk slides and any other info].

Schema evolution for relational databases: a keynote talk at the 5th International Conference on Data Management Technologies and Applications (DATA 2016), July 24 - July 26, Lisbon, Portugal [click here to watch the video of the talk at vimeo]