<p style=”text-align: justify;”>A system shell developed for modeling and execution control of knowledge-based image analysis tasks. The approach enables the flexible modeling and use of adaptive image analysis sequences to perform image-understanding tasks in complex, varying and knowledge- intensive domains such as medical image understanding. The shell supports the construction of two separated knowledge bases. The first containing knowledge about image processing algorithms and their parameters and the second containing information about image contents. This allows domain-specific control and validation. The image analysis process is controlled through an inference engine, which is based on a configuration system. The shell is called Cyclops System and is presently being used on the development of medical imaging applications.</p>
<p style=”text-align: justify;”>One of the main challenges in image understanding is to develop a flexible, adaptable system, capable of performing complex image analysis tasks and of extracting information from varying scenes or images. After three decades research, there is still no general computer vision approach, capable to provide the technology to develop a general-purpose vision system, able to deal with different tasks and images, like the human vision apparatus. In Computer Vision, the state-of-art is still that we have sorts of different, very specialized algorithms for edge-detection, segmentation, filtering, blurring and so on. If we intend to perform some image analysis task, we have to carefully choose, combine and parameterize some of those algorithms adequately. This restricts Computer Vision solutions to narrow application domains and even there, we cannot be sure that some combination of algorithms will perform in a stable and reliable manner for all images of a given category. We can identify two kinds of application domains, where existing image analysis algorithms could, in theory, be used to develop solutions: Domains where the analysis task is simple, the image acquisition is done under controlled conditions and the objects appearing in the images are known before.</p>
<p style=”text-align: justify;”>The prototypical example of such an application domain is Quality Control. In such applications, the images look almost equal, images are taken under strongly controlled conditions and images are oversimplified through filters, since we are looking for some special image characteristics. For applications like these, present technology enables us to develop specific applications from pipelines of algorithms with hard-wired parameters that work reliable (e.g. find cracks in vitrified tiles). Application domains where we have images all belonging to the same category and where there is one or a group of tasks to be performed. Typical application domains here are medical image analysis, remote sensing and content-based search in large image databases. Since we have no generic tasks, we need no generic solutions and some specific algorithms put together should solve the problem also here.</p>
<p style=”text-align: justify;”>Everybody who already tried to develop an application for such a domain described in 2., knows that this does not work. There are variations between images, even if they seem, to the human eye, belong to the same category of images and seem to be very similar: contrast characteristics of satellite images are different due to weather; tissue density from different patients differs between X-ray images; tissue appears different in MR-images if slice thickness taken was different, \dots . These variations can be strong enough to force a segmentation algorithm to produce wrong segments or an edge detection algorithm to produce incomplete borders with some set of parameters that worked for the image before. In this paper we propose and describe the implementation of an approach that solves problems that occur in domains like those of the applications exemplified above. These are domains where the tasks to be performed are always similar but much more complex than the quality control tasks, like measuring aneurysms in CT images, finding tissue suspected to be cancerous in MR-mammographies, finding all bridges in a satellite image or finding all images with sunsets in an image database.</p>
<p style=”text-align: justify;”>The approach here described was developed to be used as an expert system shell for the implementation of image analysis applications. The further refinement of both the model and the implementation of this expert system shell is one of the main tasks of the Cyclops Project, an international cooperation R\&D project existing. In order to provide feedback for the model, we are using the system for the development of several different medical imaging applications together with medical partners.
<strong>
1. Image Analysis as a Configuration Process </strong></p>
<p style=”text-align: justify;”>In the human brain the recognition and interpretation of one or several images is divided into two very hard to separate processes. The first process is the analysis of sensor data and the second is a higher intellectual process, which takes into account knowledge about possible objects in an image. In the described Cyclops system both processes have their own knowledge bases.</p>
<p style=”text-align: justify;”>knowledge base about image processing methods
knowledge base about domain objects</p>
<p style=”text-align: justify;”>Image analysis is a complex process, in which image processing routines and domain dependent interpretation steps often alternate. We now explain a simple scenario of such problems. After an initial preprocessing (for example image segmentation) a hypothesis about the image content is generated based on the domain knowledge. To confirm or reject this hypothesis several typical properties of the image must be examined, for example the segment size. This properties are stored in the domain knowledge base and create special expectations. For processing this expectations or hypothesis, algorithms must be selected based on the knowledge of the functionality of this operators. This kind of working can be iterated and the user can be asked for a judgment of the results. A very important point is to find the right configuration of the algorithm sequence by taking into account the user interactions. The user should only be asked for advise if his expertise is essential for solving the analysis problem.</p>
<p style=”text-align: justify;”>All this thoughts show that image analysis is a configuration problem, especially a configuration of an image processing process with finally an image interpretation at the end. Configuration tasks are traditionally marked off planning tasks by the fact that there is only a limited number of configuration steps,which can be chosen from a limited number of possibilities. The number of parameter sets for a configuration can be unlimited. We can find such situations often in technical domains and processes and the image analysis situation has this properties. Because of thoughts the CYCLOPS system is based on a configuration system for technical domains called IDAX .IDAX is based on the truth maintenance system REDUX. Several modifications have been made to the system to adapt it for the image analysis task. The synthesis process for a configuration consist in the creation of an object with specific demands.</p>
<p style=”text-align: justify;”>Every additional demand must be noticed for the synthesis process. Normally such conditions are modeled as constrains, sometimes as simple logical conditions. But for our problem it is completely different. The main requirement for the image processing task is that from the result the meaning of the image is visible, either for the computer or a person. In the last case the main task of the assisting system is to direct the attention of the user to the important features of the image. One problem in the image processing domain is that the description of the algorithms and the results are not stable enough to fulfill hard constraints or logical conditions. For controlling the behavior of a image analysis sequence the execution of the image analysis operators is needed and often the user must judge if the results are satisfactory. In technical domains, for example the configuration of a circuit with special properties in electrical engineering, such a control process is not needed because the propagation of the constraints during the configuration guarantee an useful result. In the image analysis domain this is not the case, even if all constraints of the configuration of an analysis process are fulfilled there can be a bad result, caused for example by variations in the image quality. If the result of an executed configuration is not good some parts of it must be refined, but a lot of results of the first execution can be reused and only some image operators are executed a second time.</p>
<p style=”text-align: justify;”><strong>2. System Architecture </strong></p>
<p style=”text-align: justify;”>In Figure the architecture of the Cyclops system is presented. The image pool is a collection of image processing methods. The image archive is based on a SQL-database. The other components are described in the following subsections.</p>
<p style=”text-align: justify;”>The execution control system: the domain knowledge system, the static knowledge base, the expectation matching component, the image archive the operator pool.
The complete knowledge about the area of image processing is stored in the static knowledge base of the configuration system. The knowledge is completely divided into concrete and abstract domain objects, which are connected by relations. With the help of this relations a concept hierarchy could be described, which is used as an orientation for the dynamical configuration process. The objects of the image processing knowledge base could be modeled with the help of the following aspects. With the aid of relations the dependencies between domain objects could be described. A taxonomical hierarchy of objects could be described by the Class-Of/Is-A relations and the compositional hierarchy by the Has-Parts/Part-Of relation. For the connections between abstract objects and technical realization a Has-Implementation relation can be used. The unary Execution relation is an extension to declare that an abstract operator is assigned for execution. With this relation concrete execution points can be declared, which are used after the complete configuration of a part process.</p>
<p style=”text-align: justify;”>Attributes describe the properties of objects. In the Cyclops system there a four groups of attributes with specific meanings input attributes output attributes parameter configurations attributes. A type is assigned to every attribute. The types of the parameter attributes normally are enumeration types and declare the set of possible operators for the determination of the parameter values. The types of the input and output attributes have a special meaning. Often image operator implementations use a specific data format and for the next step in the operator sequence an data conversion is needed. In the Cyclops system this conversions are added automatic during the dynamic configuration process by matching the types of the input and output attributes of the implementations in a sequence. Constraints are used for the modeling of restricting dependencies between objects. Preference rules allow the definition of domain dependent knowledge for decisions during the configuration process. The rules can be used for the heuristic control of configuration steps or describe a local quality criterion of the domain.</p>
<p style=”text-align: justify;”>With the phase mechanism of the static knowledge base the configuration process can be divided into a series of phases. The actual phase provides a filter for the sorting of the goals on the agenda of the inference engine. bindings are needed for the modeling of the data flow between image analysis operators. They connect attributes of different operators. We must distinguish between interface and flow bindings. Interface bindings describe connections between attributes of an aggregate and the attributes of a component. Flow bindings describe the connection between the output of an operator and the input of the following operator. Image operators can have one ore several implementations. The functionality of an operator is determined during the modeling of image analysis application. With the implementation the type of the execution call must be declared. The type and format of the call is modeled in the properties. For the execution Cyclops can use UNIX program calls, functions in an dynamic C library and Smalltalk block closures.</p>
<p style=”text-align: justify;”><strong>3. The Inference engine </strong></p>
<p style=”text-align: justify;”>The inference engine is based on the IDAX system which is an assistant system for the configuration of technical domains. I uses similar concepts like PLAKON system Between the classical configuration and the configuration of image analysis processes there are some differences:</p>
<p style=”text-align: justify;”>The description of a process is not only containing the decomposition in subprocesses or the selection of the possible implementations. The determination of the data flow between the subprocesses is also needed. Decisions during the configuration process sometimes only can be made after the complete configuration of a subprocess and the execution of the operators. For getting and controlling the results. The execution of the complete image sequence configuration with the input data is needed. Because the execution of image operators often is time consuming a correct configuration is needed To configurate means the composition of well known components to a complex thing. The problem is to find the right composition which satisfies the needs of the user and the domain constraints. In the image analysis domain there are existing additional implicit demands to the results, that can not be controlled by the propagation of constraints. Tests with the complete configurated object must be performed.
The configuration process in Cyclops follows the principle of iterative refinement. The process starts with an initial abstract object which is assigned to the goal of the configuration. By comparing this dynamical object with the fitting object in the static knowledge base operators for the processing of the goal are found. This operators for the handling of the goal can be divided into the following task classes:</p>
<p style=”text-align: justify;”>Specialization along the Is-A relation Decomposition along the Part-Of relation Parametrization, the determination of an attribute value ore the selection of a technical implementation Execution of a subconfiguration for the control of its behavior By treating a goal with the help of an operator the object is instantiated and a set of new subgoals is created. All not treated goals are administrated on an agenda. For every goal a set of operators can be chosen. This operators are the conflict set for the treatment of the goal. For the execution of a configuration step two decisions must be made. The first is the selection of the next goal and the second is the selection of one operator to reach the goal. The selection of an operator can be influenced by preference rules.</p>
<p style=”text-align: justify;”>All the decisions made during the configuration are stored in an domain specific truth maintenance system based on REDUX. With the help of the TMS all decisions about the exclusion of a operator during the treatment of a goal can be managed. This is very important for the revision mechanism of the system.</p>
<p style=”text-align: justify;”><strong>4. The Execution Control </strong></p>
<p style=”text-align: justify;”>The execution control component is responsible for the concrete execution of the elementary image processing routines. It is an interface between the inference engine and the execution of the image analysis sequence. The inference engine is creating an execution request structure of a completed configuration subtask for the execution control component. This component is following the principle of the data flow graph.</p>
<p style=”text-align: justify;”><strong><a title=”DomainKnoledgeBase” name=”DomainKnoledgeBase”></a>
5. Domain Knoledge Base </strong></p>
<p style=”text-align: justify;”>Cyclops supports a Domain Database and allows the instantiation of configuration variables with domain knowledge. These variables will typically be parameters from image operators (the underlying image analysis processes) that will hold a value, a texture, a shape description or a position of some searched structure to be used during analysis or during expectation matching. This domain knowledge is intended to be used by the image operators.</p>
<p style=”text-align: justify;”>For the development of domain databases, Cyclops provides its own tool, NeuralTaxon, which is shortly described below. It is possible, and necessary indeed, to use any other external database for accessing domain knowledge.
During the configuration process, the reduction of a DomainKnowledgeGoal can produce a query to any chosen domain database. Results from previous image analysis tasks stored in configuration variables can also be used to generate queries. This enables a modeled application to use achieved results to direct its search for domain knowledge, which can be used to parameterize further analysis steps. Queries, together with their bindings to configuration variables can be modeled in the image analysis database.</p>
<p style=”text-align: justify;”>NeuralTaxon is the name of the domain database-modeling tool included as part of Cyclops. NeuralTaxon supports the modeling of knowledge in form of hierarchical bayesian semantic networks and was developed specially to provide support for image analysis tasks. Information, which can be represented in the Neural-Taxon system, goes beyond the information in a semantic network. Not only concepts and their attributes are represented, but also the respective frequency of the occurrence of attribute-value pairs and geometric information can be stored. NeuralTaxon supports full inheritance through the concept hierarchy and also queries based on incomplete information. Using this tool we developed a domain database describing a digital brain atlas NaKS – Neuroanatomical Knowledge Source – which is intended to be used in applications in the medical imaging areas of brain stroke and neurocysticercosis presently under development.
Cyclops allows the setting of automated control points (ExpectationMatchingGoals) on the modeling of an application in the image processing database. These control points allow consistency checking. We call automatically consistence-checkable results from image processing sequences expectations.
From the point of view of the modeling of image processing, there are two distinct important aspects of consistency checking of results:
a) expectation generation and matching and
b) backtracking, reconfiguring and reexecuting involved tasks.</p>
<p style=”text-align: justify;”>Expectations are handled as normal configuration variable values that are checked against results through abstract image operators modeled as ExpectationMatchers in the database. Expectations can be values as simple as TRUE and modeled directly in the image processing database, can also be complex result descriptions generated by a query to a domain database. An ExpectationMatching goal is part of the modeling and is configured like any other image analysis sub-sequence. It is configured and executed only after all dependent analysis (sub-) tasks are finished. If an ExpectationMatching detects an error (e.g. areas suspect of being cancer in MR-mammographies are outside of the breasts), knowledge-based backtracking will take place. If the developer did not model any special behavior for backtracking, Cyclops will use optimality relationships stored in the image processing database and try to retract some executed step and replace it by the least suboptimal. The JTMS will be used to keep track of still valid results and avoid blind reprocessing. Cyclops allows also the modeling of special relations between operators, parameters, abstract operators and results to guide the retraction of configuration operators in case of unmatched expectations. The developer can: a) model which parameters or implementations are to be preferentially changed in some context, b) how much trials are to be made with some variable before another is tried and c) if some variable is only to be changed in combination with another, even if not modeled in the constraint system. The JTMS is used to keep track of all modifications an also to avoid the reprocessing of some already existing partial result.</p>
<p style=”text-align: justify;”>The MAMMALYZER II presents a diagnosis support image processing system which preprocesses dynamic MR-Mammographs (MRM) for easier and less erroneous analyzes by the medical practitioner. The breast images are provided by a special MRI sequence which is based on the work of . It uses the property that malignant tumors need a much higher perfusion than healthy tissue. Making use of a contrast-agent such as GDTA, the need for higher perfusion results in a faster accumulation of contrast agent in those areas. A comparison of the observed signal-increase ratios between different MRM-volumes gives an indication on the healthieness of the shown tissue.</p>
<p style=”text-align: justify;”>The first prototype of MAMMALYZER was structured as a pipeline of different image-processing filters controlled by a graphical user interface. This pipeline can be interactively controlled by the physician, who chooses one of two processing pipes. One for fast results and one for a more accurate analysis. A problem of the first prototype was, that most of the parameters for both matching and marking modules depend on image characteristics such as TE, TR, slice thickness, field of view, and image resolution and also on the extent of motion artifacts. A set of standard parameters was developed that works stable with the standard MRM sequences of the 1.0 T scanner of the former partner hospital in Kaiserslautern. These parameters have to be modified if one uses images from another scanner or images with another acquisition protocol. For this reason, the analysis sequences are modeled with the Cyclops system. The aim of modeling the MRM analysis under this expert system is to automatically use the results of a previous calculation of MRM-inter-volume disparities and knowledge about interrelations between image parameters and module parameters for choosing optimal analysis sequences and module parameters. Thus, users do not have to perform the task of choosing between different analysis sequences and lots of parameters, since the sequence is automatically chosen and parameterized by Cyclops which communicates with MAMMALYZER II.</p>
<p style=”text-align: justify;”>Figure shows the a simplified view of the configuration of the mamma analysis process. The abstract goal ImageProcessing is specialized to the goal MammaProcessing, which is then decomposed to the subgoals MammaPreProcessing, MammaMainProcessing and MammaExpectation. The system automatically recognizes the type of sequence and chooses appropriate image operators and parameters. After execution of the configured pipeline the expectation matching module controls the process results and decides whether they have an acceptable quality or pipeline parts must be changed. With the aid of JTMS-structure only the responsible operators, and those operators using their results, must be refined.</p>
<p style=”text-align: justify;”>Cyclops is a very powerful expert system shell for the modeling and execution of image analysis applications in complex domains. Its has proved its performance in different applications in the field of medical image analysis. Cyclops provides strong knowledge representation schemes for two kinds of knowledge. The image processing knowledge which is modeled in the static knowledge base of the configuration system and the domain knowledge which is used to solve domain dependent questions.</p>
<p style=”text-align: justify;”>In the future the shell will be improved in several fields. One important is the development of easy to use knowledge acquisition tools. The second field is the development of an approach to deal with unexpected results of an image processing sequence even if all configuration constraints are fulfilled. A set of new applications using the Cyclops system is currently under development and will show the big advantages of the configuration based approach.</p>
<p style=”text-align: justify;”>One of the main challenges in image understanding is to develop a flexible, adaptable system, capable of performing complex image analysis tasks and of extracting information from varying scenes or images. After three decades research, there is still no general computer vision approach, capable to provide the technology to develop a general-purpose vision system, able to deal with different tasks and images, like the human vision apparatus. In Computer Vision, the state-of-art is still that we have sorts of different, very specialized algorithms for edge-detection, segmentation, filtering, blurring and so on. If we intend to perform some image analysis task, we have to carefully choose, combine and parameterize some of those algorithms adequately. This restricts Computer Vision solutions to narrow application domains and even there, we cannot be sure that some combination of algorithms will perform in a stable and reliable manner for all images of a given category. We can identify two kinds of application domains, where existing image analysis algorithms could, in theory, be used to develop solutions: Domains where the analysis task is simple, the image acquisition is done under controlled conditions and the objects appearing in the images are known before.</p>
<p style=”text-align: justify;”>The prototypical example of such an application domain is Quality Control. In such applications, the images look almost equal, images are taken under strongly controlled conditions and images are oversimplified through filters, since we are looking for some special image characteristics. For applications like these, present technology enables us to develop specific applications from pipelines of algorithms with hard-wired parameters that work reliable (e.g. find cracks in vitrified tiles). Application domains where we have images all belonging to the same category and where there is one or a group of tasks to be performed. Typical application domains here are medical image analysis, remote sensing and content-based search in large image databases. Since we have no generic tasks, we need no generic solutions and some specific algorithms put together should solve the problem also here.</p>
<p style=”text-align: justify;”>Everybody who already tried to develop an application for such a domain described in 2., knows that this does not work. There are variations between images, even if they seem, to the human eye, belong to the same category of images and seem to be very similar: contrast characteristics of satellite images are different due to weather; tissue density from different patients differs between X-ray images; tissue appears different in MR-images if slice thickness taken was different, \dots . These variations can be strong enough to force a segmentation algorithm to produce wrong segments or an edge detection algorithm to produce incomplete borders with some set of parameters that worked for the image before. In this paper we propose and describe the implementation of an approach that solves problems that occur in domains like those of the applications exemplified above. These are domains where the tasks to be performed are always similar but much more complex than the quality control tasks, like measuring aneurysms in CT images, finding tissue suspected to be cancerous in MR-mammographies, finding all bridges in a satellite image or finding all images with sunsets in an image database.</p>
<p style=”text-align: justify;”>The approach here described was developed to be used as an expert system shell for the implementation of image analysis applications. The further refinement of both the model and the implementation of this expert system shell is one of the main tasks of the Cyclops Project, an international cooperation R\&D project existing. In order to provide feedback for the model, we are using the system for the development of several different medical imaging applications together with medical partners.
<strong>
1. Image Analysis as a Configuration Process </strong></p>
<p style=”text-align: justify;”>In the human brain the recognition and interpretation of one or several images is divided into two very hard to separate processes. The first process is the analysis of sensor data and the second is a higher intellectual process, which takes into account knowledge about possible objects in an image. In the described Cyclops system both processes have their own knowledge bases.</p>
<p style=”text-align: justify;”>knowledge base about image processing methods
knowledge base about domain objects</p>
<p style=”text-align: justify;”>Image analysis is a complex process, in which image processing routines and domain dependent interpretation steps often alternate. We now explain a simple scenario of such problems. After an initial preprocessing (for example image segmentation) a hypothesis about the image content is generated based on the domain knowledge. To confirm or reject this hypothesis several typical properties of the image must be examined, for example the segment size. This properties are stored in the domain knowledge base and create special expectations. For processing this expectations or hypothesis, algorithms must be selected based on the knowledge of the functionality of this operators. This kind of working can be iterated and the user can be asked for a judgment of the results. A very important point is to find the right configuration of the algorithm sequence by taking into account the user interactions. The user should only be asked for advise if his expertise is essential for solving the analysis problem.</p>
<p style=”text-align: justify;”>All this thoughts show that image analysis is a configuration problem, especially a configuration of an image processing process with finally an image interpretation at the end. Configuration tasks are traditionally marked off planning tasks by the fact that there is only a limited number of configuration steps,which can be chosen from a limited number of possibilities. The number of parameter sets for a configuration can be unlimited. We can find such situations often in technical domains and processes and the image analysis situation has this properties. Because of thoughts the CYCLOPS system is based on a configuration system for technical domains called IDAX .IDAX is based on the truth maintenance system REDUX. Several modifications have been made to the system to adapt it for the image analysis task. The synthesis process for a configuration consist in the creation of an object with specific demands.</p>
<p style=”text-align: justify;”>Every additional demand must be noticed for the synthesis process. Normally such conditions are modeled as constrains, sometimes as simple logical conditions. But for our problem it is completely different. The main requirement for the image processing task is that from the result the meaning of the image is visible, either for the computer or a person. In the last case the main task of the assisting system is to direct the attention of the user to the important features of the image. One problem in the image processing domain is that the description of the algorithms and the results are not stable enough to fulfill hard constraints or logical conditions. For controlling the behavior of a image analysis sequence the execution of the image analysis operators is needed and often the user must judge if the results are satisfactory. In technical domains, for example the configuration of a circuit with special properties in electrical engineering, such a control process is not needed because the propagation of the constraints during the configuration guarantee an useful result. In the image analysis domain this is not the case, even if all constraints of the configuration of an analysis process are fulfilled there can be a bad result, caused for example by variations in the image quality. If the result of an executed configuration is not good some parts of it must be refined, but a lot of results of the first execution can be reused and only some image operators are executed a second time.</p>
<p style=”text-align: justify;”><strong>2. System Architecture </strong></p>
<p style=”text-align: justify;”>In Figure the architecture of the Cyclops system is presented. The image pool is a collection of image processing methods. The image archive is based on a SQL-database. The other components are described in the following subsections.</p>
<p style=”text-align: justify;”>The execution control system: the domain knowledge system, the static knowledge base, the expectation matching component, the image archive the operator pool.
The complete knowledge about the area of image processing is stored in the static knowledge base of the configuration system. The knowledge is completely divided into concrete and abstract domain objects, which are connected by relations. With the help of this relations a concept hierarchy could be described, which is used as an orientation for the dynamical configuration process. The objects of the image processing knowledge base could be modeled with the help of the following aspects. With the aid of relations the dependencies between domain objects could be described. A taxonomical hierarchy of objects could be described by the Class-Of/Is-A relations and the compositional hierarchy by the Has-Parts/Part-Of relation. For the connections between abstract objects and technical realization a Has-Implementation relation can be used. The unary Execution relation is an extension to declare that an abstract operator is assigned for execution. With this relation concrete execution points can be declared, which are used after the complete configuration of a part process.</p>
<p style=”text-align: justify;”>Attributes describe the properties of objects. In the Cyclops system there a four groups of attributes with specific meanings input attributes output attributes parameter configurations attributes. A type is assigned to every attribute. The types of the parameter attributes normally are enumeration types and declare the set of possible operators for the determination of the parameter values. The types of the input and output attributes have a special meaning. Often image operator implementations use a specific data format and for the next step in the operator sequence an data conversion is needed. In the Cyclops system this conversions are added automatic during the dynamic configuration process by matching the types of the input and output attributes of the implementations in a sequence. Constraints are used for the modeling of restricting dependencies between objects. Preference rules allow the definition of domain dependent knowledge for decisions during the configuration process. The rules can be used for the heuristic control of configuration steps or describe a local quality criterion of the domain.</p>
<p style=”text-align: justify;”>With the phase mechanism of the static knowledge base the configuration process can be divided into a series of phases. The actual phase provides a filter for the sorting of the goals on the agenda of the inference engine. bindings are needed for the modeling of the data flow between image analysis operators. They connect attributes of different operators. We must distinguish between interface and flow bindings. Interface bindings describe connections between attributes of an aggregate and the attributes of a component. Flow bindings describe the connection between the output of an operator and the input of the following operator. Image operators can have one ore several implementations. The functionality of an operator is determined during the modeling of image analysis application. With the implementation the type of the execution call must be declared. The type and format of the call is modeled in the properties. For the execution Cyclops can use UNIX program calls, functions in an dynamic C library and Smalltalk block closures.</p>
<p style=”text-align: justify;”><strong>3. The Inference engine </strong></p>
<p style=”text-align: justify;”>The inference engine is based on the IDAX system which is an assistant system for the configuration of technical domains. I uses similar concepts like PLAKON system Between the classical configuration and the configuration of image analysis processes there are some differences:</p>
<p style=”text-align: justify;”>The description of a process is not only containing the decomposition in subprocesses or the selection of the possible implementations. The determination of the data flow between the subprocesses is also needed. Decisions during the configuration process sometimes only can be made after the complete configuration of a subprocess and the execution of the operators. For getting and controlling the results. The execution of the complete image sequence configuration with the input data is needed. Because the execution of image operators often is time consuming a correct configuration is needed To configurate means the composition of well known components to a complex thing. The problem is to find the right composition which satisfies the needs of the user and the domain constraints. In the image analysis domain there are existing additional implicit demands to the results, that can not be controlled by the propagation of constraints. Tests with the complete configurated object must be performed.
The configuration process in Cyclops follows the principle of iterative refinement. The process starts with an initial abstract object which is assigned to the goal of the configuration. By comparing this dynamical object with the fitting object in the static knowledge base operators for the processing of the goal are found. This operators for the handling of the goal can be divided into the following task classes:</p>
<p style=”text-align: justify;”>Specialization along the Is-A relation Decomposition along the Part-Of relation Parametrization, the determination of an attribute value ore the selection of a technical implementation Execution of a subconfiguration for the control of its behavior By treating a goal with the help of an operator the object is instantiated and a set of new subgoals is created. All not treated goals are administrated on an agenda. For every goal a set of operators can be chosen. This operators are the conflict set for the treatment of the goal. For the execution of a configuration step two decisions must be made. The first is the selection of the next goal and the second is the selection of one operator to reach the goal. The selection of an operator can be influenced by preference rules.</p>
<p style=”text-align: justify;”>All the decisions made during the configuration are stored in an domain specific truth maintenance system based on REDUX. With the help of the TMS all decisions about the exclusion of a operator during the treatment of a goal can be managed. This is very important for the revision mechanism of the system.</p>
<p style=”text-align: justify;”><strong>4. The Execution Control </strong></p>
<p style=”text-align: justify;”>The execution control component is responsible for the concrete execution of the elementary image processing routines. It is an interface between the inference engine and the execution of the image analysis sequence. The inference engine is creating an execution request structure of a completed configuration subtask for the execution control component. This component is following the principle of the data flow graph.</p>
<p style=”text-align: justify;”><strong><a title=”DomainKnoledgeBase” name=”DomainKnoledgeBase”></a>
5. Domain Knoledge Base </strong></p>
<p style=”text-align: justify;”>Cyclops supports a Domain Database and allows the instantiation of configuration variables with domain knowledge. These variables will typically be parameters from image operators (the underlying image analysis processes) that will hold a value, a texture, a shape description or a position of some searched structure to be used during analysis or during expectation matching. This domain knowledge is intended to be used by the image operators.</p>
<p style=”text-align: justify;”>For the development of domain databases, Cyclops provides its own tool, NeuralTaxon, which is shortly described below. It is possible, and necessary indeed, to use any other external database for accessing domain knowledge.
During the configuration process, the reduction of a DomainKnowledgeGoal can produce a query to any chosen domain database. Results from previous image analysis tasks stored in configuration variables can also be used to generate queries. This enables a modeled application to use achieved results to direct its search for domain knowledge, which can be used to parameterize further analysis steps. Queries, together with their bindings to configuration variables can be modeled in the image analysis database.</p>
<p style=”text-align: justify;”>NeuralTaxon is the name of the domain database-modeling tool included as part of Cyclops. NeuralTaxon supports the modeling of knowledge in form of hierarchical bayesian semantic networks and was developed specially to provide support for image analysis tasks. Information, which can be represented in the Neural-Taxon system, goes beyond the information in a semantic network. Not only concepts and their attributes are represented, but also the respective frequency of the occurrence of attribute-value pairs and geometric information can be stored. NeuralTaxon supports full inheritance through the concept hierarchy and also queries based on incomplete information. Using this tool we developed a domain database describing a digital brain atlas NaKS – Neuroanatomical Knowledge Source – which is intended to be used in applications in the medical imaging areas of brain stroke and neurocysticercosis presently under development.
Cyclops allows the setting of automated control points (ExpectationMatchingGoals) on the modeling of an application in the image processing database. These control points allow consistency checking. We call automatically consistence-checkable results from image processing sequences expectations.
From the point of view of the modeling of image processing, there are two distinct important aspects of consistency checking of results:
a) expectation generation and matching and
b) backtracking, reconfiguring and reexecuting involved tasks.</p>
<p style=”text-align: justify;”>Expectations are handled as normal configuration variable values that are checked against results through abstract image operators modeled as ExpectationMatchers in the database. Expectations can be values as simple as TRUE and modeled directly in the image processing database, can also be complex result descriptions generated by a query to a domain database. An ExpectationMatching goal is part of the modeling and is configured like any other image analysis sub-sequence. It is configured and executed only after all dependent analysis (sub-) tasks are finished. If an ExpectationMatching detects an error (e.g. areas suspect of being cancer in MR-mammographies are outside of the breasts), knowledge-based backtracking will take place. If the developer did not model any special behavior for backtracking, Cyclops will use optimality relationships stored in the image processing database and try to retract some executed step and replace it by the least suboptimal. The JTMS will be used to keep track of still valid results and avoid blind reprocessing. Cyclops allows also the modeling of special relations between operators, parameters, abstract operators and results to guide the retraction of configuration operators in case of unmatched expectations. The developer can: a) model which parameters or implementations are to be preferentially changed in some context, b) how much trials are to be made with some variable before another is tried and c) if some variable is only to be changed in combination with another, even if not modeled in the constraint system. The JTMS is used to keep track of all modifications an also to avoid the reprocessing of some already existing partial result.</p>
<p style=”text-align: justify;”>The MAMMALYZER II presents a diagnosis support image processing system which preprocesses dynamic MR-Mammographs (MRM) for easier and less erroneous analyzes by the medical practitioner. The breast images are provided by a special MRI sequence which is based on the work of . It uses the property that malignant tumors need a much higher perfusion than healthy tissue. Making use of a contrast-agent such as GDTA, the need for higher perfusion results in a faster accumulation of contrast agent in those areas. A comparison of the observed signal-increase ratios between different MRM-volumes gives an indication on the healthieness of the shown tissue.</p>
<p style=”text-align: justify;”>The first prototype of MAMMALYZER was structured as a pipeline of different image-processing filters controlled by a graphical user interface. This pipeline can be interactively controlled by the physician, who chooses one of two processing pipes. One for fast results and one for a more accurate analysis. A problem of the first prototype was, that most of the parameters for both matching and marking modules depend on image characteristics such as TE, TR, slice thickness, field of view, and image resolution and also on the extent of motion artifacts. A set of standard parameters was developed that works stable with the standard MRM sequences of the 1.0 T scanner of the former partner hospital in Kaiserslautern. These parameters have to be modified if one uses images from another scanner or images with another acquisition protocol. For this reason, the analysis sequences are modeled with the Cyclops system. The aim of modeling the MRM analysis under this expert system is to automatically use the results of a previous calculation of MRM-inter-volume disparities and knowledge about interrelations between image parameters and module parameters for choosing optimal analysis sequences and module parameters. Thus, users do not have to perform the task of choosing between different analysis sequences and lots of parameters, since the sequence is automatically chosen and parameterized by Cyclops which communicates with MAMMALYZER II.</p>
<p style=”text-align: justify;”>Figure shows the a simplified view of the configuration of the mamma analysis process. The abstract goal ImageProcessing is specialized to the goal MammaProcessing, which is then decomposed to the subgoals MammaPreProcessing, MammaMainProcessing and MammaExpectation. The system automatically recognizes the type of sequence and chooses appropriate image operators and parameters. After execution of the configured pipeline the expectation matching module controls the process results and decides whether they have an acceptable quality or pipeline parts must be changed. With the aid of JTMS-structure only the responsible operators, and those operators using their results, must be refined.</p>
<p style=”text-align: justify;”>Cyclops is a very powerful expert system shell for the modeling and execution of image analysis applications in complex domains. Its has proved its performance in different applications in the field of medical image analysis. Cyclops provides strong knowledge representation schemes for two kinds of knowledge. The image processing knowledge which is modeled in the static knowledge base of the configuration system and the domain knowledge which is used to solve domain dependent questions.</p>
<p style=”text-align: justify;”>In the future the shell will be improved in several fields. One important is the development of easy to use knowledge acquisition tools. The second field is the development of an approach to deal with unexpected results of an image processing sequence even if all configuration constraints are fulfilled. A set of new applications using the Cyclops system is currently under development and will show the big advantages of the configuration based approach.</p>
