# started from awe/astro/docs/man_contextideas.tex

\chapter{Context - basic ideas}
===============================

\label{ctx:ideas}

%\newcommand{\awe}{{\tt AWE}}
%\newcommand{\awcontext}{{\sf context}}
%\newcommand{\mydb}{{\sf MyDB}}
\newcommand{\VPD}{{\sf VPD}}
\newcommand{\makeinput}{{\bf Make-Input}}
\newcommand{\makeresult}{{\bf Make-Result}}
\newcommand{\setproject}[1]{{\sf context.set(project=#1)}}
\newcommand{\awetable}[1]{{\sf AWE#1}}
\newcommand{\sqf}{Super QualityFlag}
%\newenvironment{comment}[1]{}{}

\section{\hrsf{Context - introduction}}
%Why context? What are the properties?
%filter sea of objects/methods
%- instruments
%- projects / surveys
%  - derivation = workflow
%    - methods
%    - qualification
%- privileges
%- omegacam configuration
%\subsection{Goal of the \aw\ \awcontext}

Given the large amount of data hosted by the \aw\ Information System and the
various ways to process the data, it is useful to filter the sea of stored
objects and methods for daily data reduction work on a particular project/task.

Normally, for particular projects/tasks only a subset of all data is used at a
time. The concept \awcontext\ facilitates this by means of tools which help to
identify existing data and mark newly created data. The concept \awcontext\
also helps in marking and identifying data reduction methods or algorithms
(TBC).

It is useful to discriminate the functionality of marking objects and methods
when {\bf creating} and when {\bf viewing, reading or querying} objects.  In
the forthcoming we will discuss these two worlds separately. 

The \awcontext\ is an integrated feature of the \awenv, and therefor
should be seen as an addition to the system, without losing existing
functionalities. It is meant to support the work of groups or projects, but
could equally well support individual enterprises, like giving a demonstration,
for example.  Objects created under a given \awcontext\ can still be accessible
by the world or other groups when decided to set such privileges. 

\subsection{Properties of the \awcontext}

In the \awenv, \awcontext\ refers to the context in which raw data,
calibration data and reduced science data are {\bf created} and {\bf viewed}
(selected). The \awcontext\ is characterized by a project; i.e.~a project
defines the values of the different properties of the \awcontext .

\begin{itemize}
\item Name - The name of the project.
\item Instrument - The instrument that is used by the project.
\item Members - The people that are responsible for the project.
\item Managers - The people that can add members to the project, change the
read privileges of data, change the project defaults, etc.
\item Default privileges - Who can initially read data, that is created as part
of the project.
\item Derivation and Workflow - Defined by project designers, involving:
   \begin{itemize}
   \item Methods (TBC - input from Project designers wanted)
   \item Qualification  
   \end{itemize}
\item Category - The category or type of project - Calibration, Science,
Survey, Virtual, Maintenance. (TBC)
\item Observing Strategy: Standard, Deep, Freq. (TBC should be standard
attribute) 
\item Observing Mode: Stare, Jitter, Dither. (TBC should be standard attribute)
\item Status - Processing status. (TBC)
\end{itemize}


\subsection{Provided functionality}
The available \awcontext\ functionality can be split into two parts.  One part
is related to the {\em creation} of objects, the other part is related to the
{\em selection} of objects.

\begin{flushleft}
{\large\bf $\bullet$ Context for make}
\end{flushleft}
%
When objects are created, some of their attributes are set according to the
current \awcontext. Most \awcontext\ attributes cannot be changed because they
are defined by the project.  The following attributes are set for created
objects.
\begin{description}
\item[\awcontext-user] Name of the user.  Identifies the user accessing the
database.  Once an object is created by a user, its user attribute cannot be
changed.
\item[\awcontext-project] Name of the project.  Identifies the project within
which the user wants to work.  The user can change the \awcontext-project
during a session.  The default is undefined.
\item[privileges] Sets whether new objects are only readable by the creator,
only readable by the members of the currently select project (by definition
includes the creator), readable by \awe\ users, or readable by anyone outside
\awe, or published in VO.  The user can change the default privileges for new objects during a
session. If not set by the user, the default privileges are the privileges of
the currently selected \awcontext-project.  To be able to delete objects it is
required that only their creator has access to them. The privileges consist 
of two flags: \verb{PRIVILEGES{ and \verb{IS_VALID{. The scheme for the changes 
of these flags is presented below. 
\item[own algorithms/task/CalFile] TBD
\end{description}

There are three types of users in \awe: 
\begin{description}
\item[aw] this ``user'' is Astro-Wise itself. This mean that some kind of activity (changes of 
quality flag, for example) is made by the process started by the use but without the involvment 
of the user itself; 
\item[aw-user] the user of Astro-Wise;
\item[PM] the Project Manager or ADMINISTRATOR of the project, 
the user with the ability to create a science-ready product from 
the content of the project.  
\end{description}

Fig.~\ref{context_activity} is an activity diagram for users of Astro-Wise. This diagram does not 
take into accout viewing operation which do not change the state of entities. 


\begin{figure}
\caption{Context activity}
\label{context_activity}
\begin{center}
\includegraphics[scale=0.55]{postscript/AW_activity.eps}
\end{center}
\end{figure}

%
For existing objects, {\em only} the privileges can be changed by users.  More
specifically, the creator of an object can specify if it is also accessible to
the members of the project to which the object belongs, or to the world, if
allowed by the definition of the project.  A tool is provided to perform this
change. This ensures that all dependent objects have their privileges changed,
which is necessary to keep the database in a consistent state. A project
manager is allowed access to all data belonging to a project, even if it was
created by one of the other users in a project.


\subsection{Status of entity}

Each entity (i.e., RawFrame, RadioCube, SourceList etc.) has a set of flags which define 
the current status of the entity and the possible changes in the status. 
These flags are: 
\begin{description}
\item[] {\bf Quality flag} set by aw processing automatically. 
\item[] {\bf PRIVILEGES flag} shows the scope of visibility of the entity.  PRIVILIGES flag 
can take following values:
\begin{description}
\item[user] Data are only readable to the user.
\item[project] Data are only readable by members of the project.
\item[astro-wise] Data are only readable by \aw users, i.e.~everyone with an
\awe account.
\item[world] Data are readable by everyone, including an anonymous user.
\item[vo] Data can be accessed via VO mechanisms.
\end{description}
\item[] {\bf Validity flag} \verb{IS_VALID{ can takes following values: 
\begin{description}
\item[0] Data are invalid.
\item[1] Data are valid and can be used by other user.
\item[2] Data are valid and of scientific quality. Data can be shared with wider community
(published in paper or VO).
\end{description}
Unlike quality flag \verb{IS_VALID{ is set always by user and reflects user's subjective 
opinion about the data. 
\end{description}


The matrix of activity shows the only possible changes in the status of entity which can be made 
by the user of the Project Manager. The element of the matrix is a combination of 
flags \verb{(PRIVILEGES,IS_VALID){.  Fig.~\ref{matrix_activity} shows all allowed states for 
entities, and Table~\ref{table_activity} describes them. Actions with designation U are allowed to
be done by the user -owner of the entity, actions with P are allowed by any user in the project and
actions wih M are allowed by the Project Manager only. 

\begin{figure}
\caption{Matrix of activity}
\label{matrix_activity}
\begin{center}
\includegraphics[scale=0.50]{postscript/AD_C2.eps}
\end{center}
\end{figure}


\begin{table}
\caption{Table of states}
\label{table_activity}
\begin{tabular}{|c|c|c|c|}
\hline
Designation & Performed by & Action & Description \\
\hline
U1 & User & $(1,1)\to(1,0)$ & The entity is invalid. \\
U2 & User & $(1,0)\to(1,1)$ & The entity is valid. \\
U3 & User & $(1,1)\to(1,2)$ & The entity is selected for publishing. \\
U4 & User & $(1,2)\to(1,1)$ & The publishing is restricted. \\
U5 & User & $(1,1)\to(2,1)$ & The entity is moved to the project scope. \\
\hline
P1 & User & $(2,1)\to(2,0)$ & The entity is invalid. \\
P2 & User & $(2,0)\to(2,1)$ & The entity is valid. \\
P3 & User & $(2,1)\to(2,2)$ & The entity is selected for publishing. \\
P4 & User & $(2,2)\to(2,1)$ & The publishing is restricted. \\
P5 & User & $(2,1)\to(3,1)$ & The entity is moved to the Astro-Wise scope. \\
P6 & User & $(3,1)\to(3,1)$ & The entity is restricted for other projects. \\
P7 & User & $(2,2)\to(3,2)$ & The preselected for publishing entity \\
   &      &              & is moved to the Astro-Wise scope. \\
P8 & User & $(3,1)\to(4,1)$ & The entity is moved to the word scope. \\
P9 & User & $(4,1)\to(3,1)$ & The entity is restricted to the world. \\
P10 & User & $(4,1)\to(4,2)$ & The entity is selected for publishing. \\
\hline
M1 & PM   & $(4,2)\to(5,2)$ & The entity is published. \\
M2 & PM   & $(5,2)\to(4,2)$ & The entity is unpublished. \\
M3 & PM   & $(1,2)\to(5,2)$ & The entity is published. \\
M4 & PM   & $(2,2)\to(5,2)$ & The entity is published. \\
\hline
\end{tabular}
\end{table}


\begin{flushleft}
{\large\bf $\bullet$ Context for view}
\end{flushleft}
%
When objects are selected, some of their attributes are checked to see if they
comply with the \awcontext\ as set by the user.  Part of these attributes just
simplify querying, because their specification in queries is no longer
required. Other attributes are used to enforce read and delete permissions.

\begin{description}
\item[instrument] WFI, PDS, OCAM, WFC, ...
\item[project] KIDS, VESUVIO, VST16, WHITEDWARFS, MYPROJECT, ...
\item[privileges] There are five levels of privileges
\end{description}

\noindent
A project can be private to a specific group of users. Only these users are
allowed to access existing data or to create new data under this project.
Unauthorized users, i.e. those not part of the project, will not have any
access to a private project.

The privileges can be used to mark data as private or public to that user.
Data that are private can be deleted by the user that owns them.  However, once
data are made public, it can no longer be deleted because that could break
history tracking for data that reference it.

%The \awcontext-project allows a user to handle large groups of data.
%With the \awcontext-project new data can be tagged to be part of a certain
%project. The same project can then be used to identify data that
%belongs to that project. The project does not have to be specified in queries
%because the \awcontext-project will take care of that automatically.
%This is most useful for science data, as raw and calibration data will be used
%by more than one project.

The \awcontext\ is used to filter data, but this filtering can also be done in
\py\ queries. It should be noted that the \awcontext\ does not replace the
use of general \py\ queries. The \awcontext\ is more powerful because it can
enforce read and write access and authorization.  The \awcontext\ is also in
effect when directly using SQL from any Oracle client. Hence, the same
privileges are taken into account and authorization is required to access data
given a certain \awcontext\ .

\subsection{What Oracle provides}\label{ctx:oracle}
In Oracle it is possible to define a Virtual Private Database (\VPD), where
access to tables can be controlled at the object-level (row-level) and at the
attribute-level (column-level).

In a \VPD, access to a table is protected by one or more security policies.  A
security policy defines a limiting condition, which is automatically applied to
each query accessing a table. The condition is given in the form of a standard
{\bf where} clause or an {\bf and} clause, which means that attributes of
a table can be used freely in the condition.  Security policies can be applied
to {\bf select}, {\bf insert}, {\bf update}, {\bf delete} and
{\bf index} commands.  For each type of command a different security policy
can be created.

Since Oracle 10g, attribute-level security policies are available, which are
only applied when a particular attribute or attributes are accessed.  In
addition, protected attributes can return {\bf NULL} if selected.

A security policy uses, in Oracle terms, an Application Context.  The
Application Context makes it possible toy define, set, and access application
attributes. These application attributes are used to separate different users
and applications. Setting of the application attributes can be enforced during
logon of a database user.  The predefined {\tt USERENV} Context already
contains attributes such as {\tt CURRENT\_USERID}, {\tt ISDBA},
{\tt OS\_USER} and the like.  Security policies are written in PL/SQL,
which, for the \awenv, are part of PL/SQL package named
{\tt AWOPER.AWSECURITY}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\subsection{Building blocks}
The \awcontext\ is implemented using a standard Oracle framework, called VPD,
as explained in section~\ref{ctx:oracle}.  First, the definitions of the
tables, that are used for internal bookkeeping of projects, are given.  These
tables are created once, during the installation of the \awe\ database.  The
contents of these tables can be modified by tools, the function of which is
explained in \ref{ctx:funcs}.

\subsubsection{\awetable{PROJECTS} -- Central table with all project definitions}
\begin{flushleft}
\begin{tabular}{|l|l|l|l|}
\hline
PROJECT & DESCRIPTION & INSTRUMENT & DEFAULT PRIVILEGES \\
\hline
KIDS & ... & OCAM & PROJECT \\
VESUVIO & ... & OCAM & PROJECT \\
MYPROJECT & ... & OCAM & USER \\
PUBLIC-WFI & ... & WFI & VO \\
\hline
\end{tabular}
\end{flushleft}
%
This table contains the definitions of all projects and has columns for:
\begin{description}
\item[PROJECT] Name of the project.
\item[DESCRIPTION] Description of the project.
\item[INSTRUMENT] Instrument that is associated with a project.
\item[DEFAULT PRIVILEGES] Privileges for newly created objects. Can have any of
the privilege levels: USER, PROJECT, ASTRO-WISE, WORLD, VO.
\end{description}
A project management tool is used to change entries in this table, e.g.~to add
a project or to change the default privileges.  The possible operations on this
table are defined in section~\ref{ctx:funcs}.
\makebox[\hsize]{\hrulefill}

\subsubsection{\awetable{PROJECTUSERS} -- Table with the users of each project}
\begin{flushleft}
\begin{tabular}{|l|l|l|}
\hline
PROJECT & USER & TYPE OF USER \\
\hline
PET & RSTRAKE & ADMINISTRATOR \\
PET & KKERCHIVAL & NORMAL \\
PET & JTIDLEY & NORMAL \\
\hline
\end{tabular}
\end{flushleft}
%
This table contains the members of each project and has columns for:
\begin{description}
\item[PROJECT] Name of a project.
\item[USER] Name of the \awe\ account.
\item[TYPE OF USER] NORMAL, ADMINISTRATOR, READONLY.
\end{description}
A project management tool is used to change entries in this table.  The manager
of a project can use this tool to add users to the project.  The possible
operations on this table are defined in section~\ref{ctx:funcs}.
\\
\makebox[\hsize]{\hrulefill}

%\subsubsection{Values of \awcontext\ properties}
%\begin{description}
%\item[PROJECT] KIDS, VESUVIO, VST16, ...
%\item[USER] Name of the \awe\ account.
%\item[PRIVILEGES] USER, PROJECT, ASTRO-WISE, WORLD, VO.
%\item[TYPE OF USER] NORMAL, ADMINISTRATOR, READONLY.
%\item[INSTRUMENT] OCAM, WFI, WFC, MDM8K, PDS, ...
%\end{description}


\subsubsection{\awcontext\ attributes that are part of each DBObject}
\begin{flushleft}
\begin{tabular}{|llllll|}
\multicolumn{6}{c}{ScienceFrame, as stored in the object tables in Oracle} \\
\hline
{\it astrom} & {\it ...} & {\it ZEROPNT} & {\bf USER} & {\bf PROJECT} & {\bf PRIVILEGES} \\
\hline
10.132, -32.434 & ... & 22.15 & JTIDLEY & VESUVIO & PROJECT \\
153.541, -49.592 & ... & 22.21 & RSTRAKE & KIDS & USER \\
... & ... & ... & ... & ... & ... \\
... & ... & ... & ... & ... & ... \\
... & ... & ... & ... & ... & ... \\
\hline
\end{tabular}
\end{flushleft}
%
Here, {\it astrom, ..., ZEROPNT} are the attributes of a
{\tt ScienceFrame} as defined in \py.  The {\bf USER},
{\bf PROJECT} and {\bf PRIVILEGES} attributes are not defined in \py,
but are used by the policy functions that are defined for \awe\ in Oracle.
{\em All} objects in the database have the {\tt USER}, {\tt PROJECT} and
{\tt PRIVILEGES} attributes. It is the responsibility of the policy function
to take them into account, or not.
\\
\makebox[\hsize]{\hrulefill}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\subsubsection{\hrsf{POLICY FUNCTIONS}}
An Oracle security policy is defined for a complete {\em class} of objects.  A
security policy is only defined for certain classes.  For RawScienceFrame,
ScienceFrame, RegriddedFrame, etc.~the security policy has to be defined.
However, for classes like Chip, Filter, Instrument a security policy is not
useful.

There are separate policy functions for combinations of operations---%
SELECT=VIEW, INSERT=CREATE, UPDATE=CHANGE, DELETE---and different kinds of
tables=persistent classes.  Each combination is now discussed separately.
\\

\begin{tabular}{|l|l|l|l|l|}
\hline
& Raw Cal & Raw Science & Reduced Cal & Reduced Science \\
\hline
SELECT & & & & \\
\hline
INSERT & & & & \\
\hline
UPDATE & \multicolumn{4}{c|}{Project member only. \sqf\ + timestamp} \\
\hline
DELETE & \multicolumn{2}{c|}{Not allowed} & \multicolumn{2}{c|}{Use Data Deletion Tool} \\
\hline
\end{tabular}

\begin{description}
\item[SELECT Raw Calibration Data]
Raw calibration data are not tied to specific projects and can always
be read by all \awe\ users.
\item[SELECT Raw Science Data]
Use current project to access data.\\
{\tt WHERE PROJECT=currently\_selected\_project}
\item[SELECT Reduced Science Data]
Use current project to access data.\\
{\tt WHERE PROJECT=currently\_selected\_project}
\item[SELECT Reduced Calibration Data]
Use current project to access data.\\
{\tt WHERE PROJECT=currently\_selected\_project}
%
\item[] \makebox[\hsize]{\hrulefill}
%
\item[INSERT Raw Calibration Data]
Raw calibration data are not tied to specific projects.
\item[INSERT Raw Science Data]
Set project to current project.\\
{\tt INSERT INTO ARAWSCIENCETABLE SET PROJECT=currently\_selected\_project}
\item[INSERT Reduced Calibration Data]
Set project to current project.\\
{\tt INSERT INTO AREDUCEDCALTABLE SET PROJECT=currently\_selected\_project}
\item[INSERT Reduced Science Data]
Set project to current project.\\
{\tt INSERT INTO AREDUCEDSCIENCETABLE SET PROJECT=currently\_selected\_project}
%
\item[] \makebox[\hsize]{\hrulefill}
%
\item[UPDATE Raw Calibration Data]
Only the \sqf\ can be changed.\\
\item[UPDATE Raw Science Data]
Only the \sqf\ can be changed.\\
{\tt WHERE PROJECT=currently\_selected\_project}
\item[UPDATE Reduced Calibration Data]
Only the timestamp and the \sqf\ can be changed.\\
{\tt WHERE PROJECT=currently\_selected\_project}
\item[UPDATE Reduced Science Data]
Only the \sqf\ can be changed.\\
{\tt WHERE PROJECT=currently\_selected\_project}
%
\item[] \makebox[\hsize]{\hrulefill}
%
\item[DELETE Raw Calibration Data] Not allowed.
\item[DELETE Raw Science Data] Not allowed.
\item[DELETE Reduced Calibration Data] Only possible with Data Deletion Tool.
\item[DELETE Reduced Science Data] Only possible with Data Deletion Tool.
\end{description}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\subsection{Functionalities}\label{ctx:funcs}
\subsubsection{CREATE PROJECT (name, description, users=[], instrument, privileges)}
\begin{description}
\item[Performed by] User.
\item[Purpose] Define a new project.
\item[Description] Create a new project {\em name} for a group of users
that is associated with the project. An instrument and default privileges
can be defined for the project.
\item[Specifies] \
  \begin{itemize}
  \item Name.
  \item A description of the project.
  \item Project users and their type (NORMAL, ADMINISTRATOR, READONLY).
  \item Instrument.
  \item Default privileges.
  \end{itemize}
%\item[Note] To make {\em all} \awe\ users member of a project, a special user
%called ANONYMOUS can be added to the list of project members.
\item[Restriction] None.
\end{description}
%
%Users can define a new project by using a \py\ command. This command also
%sets initial properties of the project.  The user who defines the project is
%automatically added to the users of the project with a type of SUPERUSER.  The
%project definition includes such things as the project name, the users and
%managers of the project, the default priviliges and the instrument used.
%\\
\makebox[\hsize]{\hrulefill}

\subsubsection{DELETE PROJECT (name)}
\begin{description}
\item[Performed by] DBA.
\item[Purpose] Delete the definition of an existing project called
\textsl{name}.
\item[Description] Project definitions that are no longer used can be deleted.
\item[Restriction] Project definitions can only be deleted if there is no data
associated with the project.
\end{description}
\makebox[\hsize]{\hrulefill}

\subsubsection{CHANGE PROJECT (privileges$|$instrument$|$drop user$|$add user)}
\begin{description}
\item[Performed by] User with appropriate privileges.
\item[Purpose] \
  \begin{itemize}
  \item Add a user to the project.
  \item Delete a user from the project.
  \item Change the type of a user.
  \item Change the instrument for a project.
  \item Change the default privileges for a project, i.e.~the privileges that
  are set for newly created objects.
  \end{itemize}
\item[Description] After a project has been defined, it is possible to add
users to it or remove users from it. It is also possible to change the type of
each project user to NORMAL, ADMINISTRATOR or READONLY.
\item[Restriction] \
  \begin{itemize}
  \item Name cannot be changed.
  \item Only users of type ADMINISTRATOR can change the project definition.
  \end{itemize}
\end{description}
\makebox[\hsize]{\hrulefill}

\subsubsection{SET PROJECT}
\begin{description}
\item[Performed by] User.
\item[Purpose] Select a project in an interactive \awe\ session or script.
\item[Description] Inside \awe\ this function is used to select the current
project. Selecting the project affects all making and querying of data that
follows.
\item[Restriction] A project can only be selected if the user is a member of
the project. If the ANONYMOUS user is member of the project, {\em all} users
can select the project.
\end{description}
\makebox[\hsize]{\hrulefill}

\subsubsection{GET PROJECT}
\begin{description}
\item[Performed by] User.
\item[Purpose] Obtain the definition of a project in an interactive \awe\
session or script.
\item[Description] Find out which group of users are taking part in a project,
what the description of a project is, what instrument is being used, etc.  This
is useful in scripts and webservices, where it is only known at runtime which
project is selected.
\item[Restriction] None.
\end{description}
\makebox[\hsize]{\hrulefill}

\subsubsection{GET LIST OF PROJECTS}
\begin{description}
\item[Performed by] User.
\item[Purpose] Get a list of all projects in an interactive \awe\ session or
script.
\item[Description] Obtain a list of all currently defined projects.  This is
useful in scripts and webservices, where it is only known at runtime which
project is selected.
\item[Restriction] None.
\end{description}
\makebox[\hsize]{\hrulefill}

\begin{comment}{
\vspace{1cm}
\noindent%
For the following items it is important to note that:
\begin{itemize}
\item
Each object in the database has a USER (the creator of the object), a PROJECT
and PRIVILEGES assigned to it.
\item
Data are grouped into two categories.
\begin{description}
\item[\makeinput] Data that serve as input for recipes, i.e.~raw calibration
and science data, calibration files, science frames, regridded frames and
configuration data.
\item[\makeresult] Data that that are the result of recipes, i.e.~calibration
files, astrometrically and photometrically calibrated (and regridded) images,
coadded images, source lists and associate lists.
\end{description}
The \makeresult\ refers to the object for which the {\tt make()} method has
been executed and \makeinput\ refers to the data that were used by the
{\tt make()} method.
As can be seen in the above definition, some \makeresult\ data
can be used as \makeinput\ for other make() operations.
\end{itemize}
%
\makebox[\hsize]{\hrulefill}
}\end{comment}


%\subsubsection{CREATE DATA FOR A PROJECT}
%\begin{description}
%\item[Performed by] User.
%\item[Purpose] Automatically associate a project with newly created data.
%\item[Description] To create data for a project, use SET PROJECT to select
%a project. From then on, the project is set automatically for all persistent
%objects that are created by the {\tt commit()} method of a persistent
%object.
%\item[Restriction] None.
%\end{description}
%\makebox[\hsize]{\hrulefill}

%\subsubsection{SELECT DATA OF A PROJECT}
%\begin{description}
%\item[Performed by] User.
%\item[Purpose] 
%\item[Description]
%\item[Restriction] None.
%\end{description}
%Data can be selected if allowed by its privilege attribute.
%Only data for which the user has read-access is visible when querying via
%\py\ or directly via SQL. The selecting is also based on the current
%project. E.g.~if a project defined an instrument, only data for that
%instrument is seen when the \awcontext\ is set for that project.
%All selections take the \awcontext-project into account, if it is set, and
%no command is needed, other than the command to select a \awcontext-project.
%\\
%\makebox[\hsize]{\hrulefill}

\subsubsection{DELETE DATA OF A PROJECT}
\begin{description}
\item[Performed by] User with appropriate privileges.
\item[Purpose] Clean up things that are no longer needed.
\item[Description] Data can be marked for removal. In that case, all data that
belong together have to be removed, because of the requirement that the
complete history of any object can be tracked.  There are two categories of
data that can be deleted:
\begin{itemize}
\item Reduced calibration data.
\item Reduced science data.
\end{itemize}
\item[Restriction]
Data can only be deleted if there are no references to it.
%Only \makeresult\ data can be deleted. Deleting \makeresult\ data implies
%deletion of all intermediate dependent objects (including files) until, but
%excluding, the \makeinput\ data.
\end{description}
\makebox[\hsize]{\hrulefill}

%\subsubsection{CHANGE THE PROJECT OF DATA}
%\begin{description}
%\item[Performed by] User with appropriate privileges.
%\item[Purpose] 
%\item[Description]
%\item[Restriction]
%Changing the \awcontext-project attribute to ``move'' data to a different
%project is allowed if certain conditions are satisfied. Only for \makeresult\ 
%data the project can be changed. Like for deletion, all dependent objects
%until---but excluding---the \makeinput\ have their project changed.
%To be able to change the \awcontext-project one has to be manager of
%both projects.
%\end{description}
%The only attributes of data that can be changed, are the \awcontext-project
%and attributes like {\tt timestamp\_start}, {\tt timestamp\_end},
%{\tt quality\_flags}. Changing other attributes is not allowed, because
%it could violate the history-tracking requirement.
%\\
%\makebox[\hsize]{\hrulefill}

\subsubsection{CHANGE THE READ PRIVILEGES OF DATA}
\begin{description}
\item[Performed by] User with appropriate privileges.
\item[Purpose] Make data visible to a different (larger) group of users.
\item[Description] The read level of the data can be either USER (only readable
by the user), PROJECT (only readable by members of the project), \aw\
(readable by \awe\ users), WORLD (readable by anyone) or VO (readable by VO
compliant clients).
\item[Restriction] The read privileges can only be changed in such a way that
they give access to the data to a larger group. Restricting privileges to a
smaller group could break history tracking and is therefore forbidden.
\end{description}
\makebox[\hsize]{\hrulefill}




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{\mydb}
\mydb\ is defined as the collection of all user data ({\bf privileges} set to
{\bf user}).
\mydb\ serves two purposes. On one hand it allows the user to
{\em create persistent objects} that are part of the shared \awe\ database.
On the other hand \mydb\ allows users to {\em add persistent classes} for
themselves, giving the user flexible database access from \py\ scripts.
\mydb\ also allows direct creation of tables, views and similar database
constructions via SQL.

Persistent classes that are defined in \mydb, can refer to the persistent
classes in the shared \awe\ database. The reverse is not true, i.e. the
persistent classes in the shared \awe\ database are not allowed to refer to
\mydb\ persistent classes.

Calibration files and reduced science images, that are created in the shared
\awe\ database with the \mydb\ context, are initially only visible to the user
that created them. At a later stage, the user can decide to make these data
visible to the rest of the community, e.g. the people participating in a
project. Each calibration file and reduced science image has a method to
perform this action. It is important to know that this operation is done
recursively and is irreversible. This keeps the shared data consistent and
complete, by ensuring that there are no references from the shared data to any
data in \mydb.



\subsection{Setting and changing \awcontext\ properties}
The following example shows how the \awcontext\ is set in \awe\
\begin{verbatim}
awe> from astro.database.Context import context
awe> context.set(instrument='OCAM')   # Operate on OmegaCAM data exclusively
awe> print len(RawScienceFrame.filename != '')
13245
awe> context.set(project='Hercules')  # Only Hercules + OmegaCAM data
awe> print len(RawScienceFrame.filename != '')
2013
awe> context.unset('project')         # Select from any project again
awe> context.unset('instrument')      # Select from any instrument again
awe> print len(RawScienceFrame.filename != '')
34767
\end{verbatim}

\noindent%
The following example shows how the \awcontext\ is set in {\tt SQL}
\begin{verbatim}
SQL> EXECUTE AWOPER.AWSECURITY.SET_INSTRUMENT('WFI');

PL/SQL procedure successfully completed.

SQL> SELECT COUNT(*) FROM "RawScienceFrame";

  COUNT(*)
----------
     10632

SQL> EXECUTE AWOPER.AWSECURITY.UNSET_INSTRUMENT();

PL/SQL procedure successfully completed.

SQL> SELECT COUNT(*) FROM "RawScienceFrame";

  COUNT(*)
----------
     20740
\end{verbatim}




\subsection{Open issues}
\begin{itemize}
\item algorithms
\item specific methods for calibration
\end{itemize}

\endinput



%\noindent%
%\begin{table}
%\begin{tabular}{llllll}
%\hline
%\awcontext & Create & Select    & Select    & Can be  \\
%           &        & Exclusive & Inclusive & Private \\
%\hline
%project    & Yes & Yes & Yes & Yes \\
%user       & Yes & Yes & Yes & Yes \\
%instrument & N/A & Yes & No  & No  \\
%category   & N/A & Yes & No  & No  \\
%strategy   & N/A & Yes & No  & No  \\
%obsmode    & N/A & Yes & Yes & No  \\
%algorithm  & Yes & Yes & Yes & ?   \\
%\hline \\
%\end{tabular}
%
%N/A - ignores the \awcontext\ setting, Yes - is allowed, No - is not allowed.
%\caption{Operations on raw data, per \awcontext\ item.}
%\end{table}
%
%\begin{table}
%\begin{tabular}{llllll}
%\hline
%\awcontext & Create & Select    & Select    & Can be  \\
%           &        & Exclusive & Inclusive & Private \\
%\hline
%project    & Yes & Yes & Yes & Yes \\
%user       & Yes & Yes & Yes & Yes \\
%instrument & N/A & Yes & No  & No  \\
%category   & N/A & Yes & No  & No  \\
%strategy   & N/A & Yes & No  & No  \\
%obsmode    & N/A & Yes & Yes & No  \\
%algorithm  & Yes & Yes & Yes & ?   \\
%\hline \\
%\end{tabular}
%
%N/A - ignores the \awcontext\ setting, Yes - is allowed, No - is not allowed.
%\caption{Operations on calibration data, per \awcontext\ item.}
%\end{table}
%\begin{table}
%\begin{tabular}{llllll}
%\hline
%\awcontext & Create & Select    & Select    & Can be  \\
%           &        & Exclusive & Inclusive & Private \\
%\hline
%project    & Yes & Yes & Yes & Yes \\
%user       & Yes & Yes & Yes & Yes \\
%instrument & N/A & Yes & No  & No  \\
%category   & N/A & Yes & No  & No  \\
%strategy   & N/A & Yes & No  & No  \\
%obsmode    & N/A & Yes & Yes & No  \\
%algorithm  & Yes & Yes & Yes & ?   \\
%\hline \\
%\end{tabular}
%
%N/A - ignores the \awcontext\ setting, Yes - is allowed, No - is not allowed.
%\caption{Operations on processed science data, per \awcontext\ item.}
%\end{table}

%\hrulefill
%\subsection{Open questions}
%\begin{itemize}
%\item Privileges (permissions) to read, write or delete data.
%\item Quality of calibration, how data was calibrated (note: may not be
%context, but more in the general query domain).
%\item Identification for common scratch space for users.
%\item A personal TEST and BASE version (only useful if it is possible to
%change the status.
%\end{itemize}
%
%\awcontext\ attributes should be as orthogonal as possible. This can avoid
%the possibility of conflicting \awcontext\ restrictions which can lead to
%answers that are confusing to the user.
%\begin{itemize}
%\item Some \awcontext\ attributes require authorisation.
%\item Some \awcontext\ attributes require the ability to change them.
%\item Some \awcontext\ attributes are only useful in combination with others.
%
%\item What \awcontext\ items are useful apart from the above?
%\item Should (some of) the \awcontext\ items be attributes of persistent classes?
%\item Should all persistent classes have all/the \awcontext\ items?
%\end{itemize}


%%% EXAMPLE BEGIN
Before going into details, take an example project.  A group of persons want to
work together to reduce data of Chamaeleon, observed with OmegaCAM, for their
pet project.  Some of the \awcontext\ properties for this project are set as
follows:
\begin{description}
% WARNING: The \tt also sets the fonts for the explanation in the next lines
\item[Name] \tt PET
\item[Description] Chamaeleon at Large
\item[Instrument] OCAM
\item[Project Members] KKERCHIVAL, RSTRAKE, JTIDLEY
\item[Project Manager] RSTRAKE
\item[Default Privileges] Only readable by Project Members
\end{description}
%
The {\tt PET} project defines all context properties. K.~Kerchival,
R.~Strake and J.~Tidley are members of the {\tt PET} project.  The project
manager is R.~Strake, who can add members to the project or make the project
data readable by everyone outside the project.  Data that is created can,
initially, only be read by the project members.


%%% OBSOLETE??? BEGIN
Therefore, the instrument \awcontext\ is set to {\tt OCAM}\\
\begin{verbatim}
context.set(instrument='OCAM')
\end{verbatim}
From now on,  all the querying ({\bf viewing} of the database will result in
data from Omegacam, and all other instruments will be hidden. Of course, all
objects {\bf created } will become OmegaCAM.  The project \awcontext\ is set to
{\tt PET}.
\begin{verbatim}
context.set(project='PET')
\end{verbatim}
%%% OBSOLETE??? END
This will set the privileges when {\bf creating} new objects; these will be
copied from a central table operated by DBA? on demand of the project
responsible.  These privileges apply when subsequently {\bf viewing} the
objects of project {\tt PET}.\\
The project responsible can decide to discriminate various privileges for
various types of objects like {\sf CalFile}s for  {\bf viewing} by other
users.  Similarly, public release can be initiated.




To derive the best possible results, people working on the {\tt PET} project
try new algorithms. The method \awcontext\ helps them to filter results that
were derived with different methods.  Because much time is spent on quality
control of the results, the qualification \awcontext\ is used to limit access
to data that were observed under exceptional seeing conditions.
%%% EXAMPLE END