Image Sequences

In the 1970s, Hans-Hellmut Nagel began recording the image sequences shown here as part of his research into image evaluation. Originally, the image sequences were available to other researchers on the website of the Institut für Algorithmen und Kognitive Systeme of the Fakultät für Informatik at the Universität Karlsruhe. This website is now only accessible via the Internet Archive (status 01.06.2023). There you can see how the site has changed from 21.02.1999 to 01.06.2023. However, the actual videos of the image sequences cannot be viewed there. For this reason, Florian Müller and Jakob Tschandl have reconstructed this website for the Deutsches Museum on behalf of the Cryptography and Security Group of the Institute of Information Security and Dependability at KIT. Here you can view and download the image sequences with a short introduction by Hans-Hellmut Nagel.

The image sequences made accessible here have been recorded/collected since the 1970s as part of a long-term university basic research project "Algorithmische Umsetzung digitisierter Bildfolgen in natürlich-sprachliche Beschreibungen der aufgezeichneten Vorgänge" ("Algorithmic conversion of digitized image sequences into natural language descriptions of the recorded processes").

Without an immediately available application, it was necessary to work with limited resources (i.e. no mainframe computer, no special - expensive - device development; instead, networked small computers on which members of the research group could easily experiment in isolation if necessary, without running the risk of significantly disturbing many other computer users in their work in the event of their own errors).

Existing technologies - e.g. video technology with cameras, analogue recording on video tapes, television monitors for viewing, including computer-generated (interim) results - had to be linked together by laboratory developments: a "small computer network for recording, digitizing, storing and evaluating video image sequences".

After digitization, this use of video technology led to a data rate (in bytes per second) and data volumes (in megabytes) that far exceeded what could be handled by the main memories of computers available at the beginning of the 1970s.

After several years of development work, we succeeded in digitally recording and analyzing video image sequences in our laboratory from the middle third of the 1970s onwards.

This meant that the development of algorithms for isolating body images based solely on the movement of the images of real bodies relative to the stationary camera became increasingly important. Closely related to this was the tracking of isolated image sections from shot to shot.

In order to prevent algorithm development from being over-adapted to the special characteristics of the image sequence being analyzed, numerous different image sequences had to be recorded and analyzed.

Over the years, this resulted in a collection of image sequences, some examples of which are shown below.

Until the beginning of the 2000s, the effort involved in recording and digitally storing image sequences was still so great that smaller laboratories in particular were interested in such data.

The image sequences accessible here originate from a website that was created under the direction of H.-H. Nagel at the Institut für Algorithmen und Kognitive Systeme (IAKS) with the aim of making video image sequences generally available for research purposes. This website was also used by external research groups to compare evaluation algorithms of different research groups on the same image sequences.

The continuous improvement and cheapening of digital technologies led to a perceived decline of interest in downloading image sequences from this website from around the year 2000 onwards¹⁾.

Hans-Hellmut Nagel, KIT, March 2024

We also created a website for the project DARVIN by Hans-Hellmut Nagel.

¹⁾ Acknowledgments

Without Holger Hellmuth from the Cryptography and Security Group of the Institute of Information Security and Dependability at KIT - a successor institute of the Institut für Algorithmen und Kognitive Systeme (IAKS) at the Universität Karlsruhe (TH) - I would not have been able to secure the extensive files of the IAKS website  "i21www...." prior to the shutdown of this website at the end of July 2023. I sincerely thank him for his help. J. Theuerkorn, head of the Technical Infrastructure Section (ATIS) at the Karlsruhe Institute of Technology (KIT), and Th. Griesbaum, Department of Informatics at KIT, have also graciously assisted me in my efforts to overcome the challenges associated with the shutdown of the former IAKS website.

R. Seising of the Deutsches Museum in Munich impresses me by his openmindedness regarding the variant of AI documented below. He earned my gratitude for his efforts to include the website "Bildfolgen" and "DARVIN" in the documentation of the IGGI project he initiated and led at the Deutsches Museum, as well as for the very pleasant cooperation.

My special thanks go to Mrs. C. Manietta, Cryptography and Security Group of the Institute of Information Security and Dependability at KIT: she tirelessly took care of my numerous concerns after my retirement; special thanks go to her for preserving my remaining funds at KIT, which were not bound by grant conditions, and for her successful efforts to be able to use these remaining funds for the design of the website at the Deutsches Museum mentioned above.

Prof. J. Mueller-Quade, head of the Cryptography and Security Group at KIT, took a keen interest in the longer-term documentation of this variant of AI and thus made its implementation possible.

A website intended for the general public of the Deutsches Museum is different from a scientific publication ("In Echt", as they say in Hamburg). In my attempts to learn this, Fl. Müller and J. Tschandl responded to my almost endless suggestions for changes with tireless patience and helpfulness.

For reasons of space, I cannot list the names of the numerous cooperating people who have assisted me over many decades in setting up and using various systems for recording, evaluating and documenting image sequences. I would also like to thank them all.

Hans-Hellmut Nagel

Karlsruher Institut for Technology, Karlsruhe, March 2024

²⁾ Durlacher-Tor Bildfolge: first used by Dieter Koller, see D. Koller: "Detektion, Verfolgung und Klassifikation bewegter Objekte in monokularen Bildfolgen am Beispiel von Straßenverkehrsszenen." Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH), Juni 1992; published in Dissertationen zur Künstlichen Intelligenz (DISKI) 13, infix-Verlag St. Augustin 1992. See also D. Koller, K. Daniilidis, T. Thorhallson, and H.-H. Nagel: "Model-Based Object Tracking in Traffic Scenes." In: G. Sandini (Ed.), "Proc. Second European Conference on Computer Vision (ECCV'92), 19-22 May 1992, Santa Margherita Ligure/Italy", Lecture Notes in Computer Science 588, Springer-Verlag Berlin, Heidelberg 1992, pp. 437-452; and D. Koller, K. Daniilidis, and H.-H. Nagel: "Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes." In: International Journal of Computer Vision 10 (1993) 257-281.

³⁾ Ettlinger-Tor Bildfolge: recorded and first analyzed by H. Kollnig, see: Henner Kollnig: "Ermittlung von Verkehrsgeschehen durch Bildfolgenauswertung." Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH), Februar 1995; published in Dissertationen zur Künstlichen Intelligenz (DISKI) 88, infix-Verlag St. Augustin 1995. See also H. Kollnig, H.-H. Nagel, and M. Otte: "Association of Motion Verbs with Vehicle Movements Extracted from Dense Optical Flow Fields." In: J.-O. Eklundh (Ed.), "Proc. 3rd European Conference on Computer Vision (ECCV '94), 2-6 May 1994, Stockholm/S", Lecture Notes in Computer Science 801 (Vol. II), Springer Berlin Heidelberg New York 1994, pp. 338-347; and Kollnig, H., and Nagel, H.-H.: "3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients." In: International Journal of Computer Vision 23:3 (1997) 283--302.

⁴⁾ Nibelungen-Platz image sequence: analyzed in Kollnig, Dissertation 1995 (s.o.), and H. Kollnig, H. Leuck and H.-H. Nagel: "Detektion und Verfolgung von Fahrzeugen in Straßenverkehrsszenen: Systematische Bewertung und Steigerung der Systemleistung." In: G. Sagerer, St. Posch, F. Kummert (Eds.): "Mustererkennung 1995, 17. DAGM-Symposium, 13.-15. September 1995, Bielefeld", Informatik aktuell, Springer-Verlag Berlin, Heidelberg, New York, u. a. 1995, pp. 303-310.

⁵⁾ Karl-Wilhelm-Straße: recorded by Klaus Fleischer, Michael Haag, Frank Heimes, and Sven Noltemeier on 23.07.1997; first results in H.-H. Nagel, T. Schwarz, H. Leuck, and M. Haag: "Tracking Turning Trucks with Trailers." In: S. Maybank and T. Tan (Eds.): "Proc. IEEE Workshop on Visual Surveillance, 2 January 1998, Bombay/India (in conjunction with ICCV '98)", pp. 65—72; and "Beginning a Transition from a Local to a More Global Point of View in Model-Based Vehicle Tracking." In: H. Burkhardt and B. Neumann (Eds.): "Proc. Fifth European Conference on Computer Vision (ECCV'98), 2-6 June 1998, Freiburg/Germany", Lecture Notes in Computer Science LNCS 1406 (Vol. I), Springer-Verlag Berlin, Heidelberg, New York 1998, pp. 812-827.

⁶⁾ Karl-Wilhelm-Straße: stationary camera installed by German Eichberger, Klaus Fleischer und Holger Leuck.