Request: Help creating a difficult view.

Hello all.

I have a table defined in sql server as follows:

ROW_ID (identity)
DEPTH_FROM Number (8,3)
DEPTH_TO Number (8,3)
COLOUR Char(10)

With typical data like:
ROW_ID DEPTH_FROM DEPTH_TO COLOUR
-------------------
1 0 5
BLUE
2 5 8
BLUE
3 8 10
RED
4 10 12
GREEN
5 12 16
GREEN

I want to create a view that will 'compress/roll up' the data so
it appears like:

DEPTH_FROM DEPTH_TO COLOUR
------------------
0 8 BLUE
8 10 RED
10 16 GREEN

I have been working on this for several days, with no luck,
any help would be appreciated. BTW: there are no overlaps
allowed in the depth_from, depth_to values.

Thanks in advance.I'll assume that the colours don't always occur at consecutive depths
otherwise you could just do this:

SELECT MIN(depth_from), MAX(depth_to), colour
FROM ColDepths
GROUP BY colour

Here's the (assumed) DDL and sample data. It helps if you include this with
posts.

CREATE TABLE ColDepths (row_id INTEGER NOT NULL UNIQUE, depth_from INTEGER
NOT NULL, depth_to INTEGER NOT NULL, colour CHAR(10) NOT NULL,
CHECK(depth_from<depth_to), PRIMARY KEY (depth_from,depth_to))

INSERT INTO ColDepths VALUES (1,0,5, 'BLUE')
INSERT INTO ColDepths VALUES (2,5,8, 'BLUE')
INSERT INTO ColDepths VALUES (3,8,10, 'RED')
INSERT INTO ColDepths VALUES (4,10,12, 'GREEN')
INSERT INTO ColDepths VALUES (5,12,16, 'GREEN')

Here's my query.

SELECT MIN(A.depth_from) AS depth_from,
MAX(A.depth_to) AS depth_to, A.colour
FROM ColDepths AS A
JOIN
(SELECT c1.row_id, MIN(C2.depth_to) AS next_depth
FROM ColDepths AS C1
LEFT JOIN ColDepths AS C2
ON C1.colour <> C2.colour
AND (C1.depth_from < C2.depth_from
OR (C1.depth_from = C2.depth_from)
AND C1.depth_to <= C2.depth_to)
GROUP BY c1.row_id) AS B
ON A.row_id = B.row_id
GROUP BY A.colour, B.next_depth

--
David Portas
----
Please reply only to the newsgroup
--|||Is this what you have in mind?

create table foo
(ROW_ID int, /* my datatypes vary from yours for my convenience
*/
DEPTH_FROM int,
DEPTH_TO int,
COLOUR Char(10))
go
insert foo values (1,0,5,'blue')
insert foo values (2,5,8,'blue')
insert foo values (3,8,10,'red')
insert foo values (4,10,12,'green')
insert foo values (5,12,16,'green')
go
select min(depth_from) as depth_from, max(depth_to) as depth_to, colour
from foo
group by colour

depth_from depth_to colour
---- ---- ----
0 8 blue
10 16 green
8 10 red

(3 row(s) affected)

You may want an ORDER BY clause, too. Order of rows returned with GROUP BY
is not guaranteed/predictable.

"Dave Pylatuk" <davep@.centurysystems.net> wrote in message
news:pZgcb.4184$1H3.311803@.news20.bellglobal.com.. .
> Hello all.
> I have a table defined in sql server as follows:
> ROW_ID (identity)
> DEPTH_FROM Number (8,3)
> DEPTH_TO Number (8,3)
> COLOUR Char(10)
> With typical data like:
> ROW_ID DEPTH_FROM DEPTH_TO COLOUR
> -------------------
> 1 0 5
> BLUE
> 2 5 8
> BLUE
> 3 8 10
> RED
> 4 10 12
> GREEN
> 5 12 16
> GREEN
> I want to create a view that will 'compress/roll up' the data so
> it appears like:
> DEPTH_FROM DEPTH_TO COLOUR
> ------------------
> 0 8 BLUE
> 8 10 RED
> 10 16 GREEN
> I have been working on this for several days, with no luck,
> any help would be appreciated. BTW: there are no overlaps
> allowed in the depth_from, depth_to values.
> Thanks in advance.|||"Dave Pylatuk" <davep@.centurysystems.net> wrote...

> I want to create a view that will 'compress/roll up' the data so
> it appears like:
> DEPTH_FROM DEPTH_TO COLOUR
> ------------------
> 0 8 BLUE
> 8 10 RED
> 10 16 GREEN

I don't use SqlServer but... wouldn't this work?

select min(depth_from), max(depth_to), colour from <tablename> group by
colour|||Testing these suggestions right now, thanks to all

"Dave Pylatuk" <davep@.centurysystems.net> wrote in message
news:pZgcb.4184$1H3.311803@.news20.bellglobal.com.. .
> Hello all.
> I have a table defined in sql server as follows:
> ROW_ID (identity)
> DEPTH_FROM Number (8,3)
> DEPTH_TO Number (8,3)
> COLOUR Char(10)
> With typical data like:
> ROW_ID DEPTH_FROM DEPTH_TO COLOUR
> -------------------
> 1 0 5
> BLUE
> 2 5 8
> BLUE
> 3 8 10
> RED
> 4 10 12
> GREEN
> 5 12 16
> GREEN
> I want to create a view that will 'compress/roll up' the data so
> it appears like:
> DEPTH_FROM DEPTH_TO COLOUR
> ------------------
> 0 8 BLUE
> 8 10 RED
> 10 16 GREEN
> I have been working on this for several days, with no luck,
> any help would be appreciated. BTW: there are no overlaps
> allowed in the depth_from, depth_to values.
> Thanks in advance.|||Hi Dave,

If there is no overlap but there can be gaps between intervals, this is the
query you want:

select DEPTH_FROM,
DEPTH_TO = (select min(DEPTH_TO)
from T T3
where DEPTH_TO not in (select DEPTH_FROM
from T T4
where T3.COLOUR = T4.COLOUR
and T3.DEPTH_FROM <>
T4.DEPTH_FROM
)
and T3.DEPTH_TO > T1.DEPTH_FROM
),
COLOUR
from T T1
where DEPTH_FROM not in (select DEPTH_TO
from T T2
where T1.COLOUR = T2.COLOUR
and T1.DEPTH_FROM <> T2.DEPTH_FROM
)
order by DEPTH_FROM

Good Luck,
Shervin

"Dave Pylatuk" <davep@.centurysystems.net> wrote in message
news:pZgcb.4184$1H3.311803@.news20.bellglobal.com.. .
> Hello all.
> I have a table defined in sql server as follows:
> ROW_ID (identity)
> DEPTH_FROM Number (8,3)
> DEPTH_TO Number (8,3)
> COLOUR Char(10)
> With typical data like:
> ROW_ID DEPTH_FROM DEPTH_TO COLOUR
> -------------------
> 1 0 5
> BLUE
> 2 5 8
> BLUE
> 3 8 10
> RED
> 4 10 12
> GREEN
> 5 12 16
> GREEN
> I want to create a view that will 'compress/roll up' the data so
> it appears like:
> DEPTH_FROM DEPTH_TO COLOUR
> ------------------
> 0 8 BLUE
> 8 10 RED
> 10 16 GREEN
> I have been working on this for several days, with no luck,
> any help would be appreciated. BTW: there are no overlaps
> allowed in the depth_from, depth_to values.
> Thanks in advance.|||"Dave Pylatuk" <davep@.centurysystems.net> wrote in message
news:pZgcb.4184$1H3.311803@.news20.bellglobal.com.. .
> Hello all.
> I have a table defined in sql server as follows:
> ROW_ID (identity)
> DEPTH_FROM Number (8,3)
> DEPTH_TO Number (8,3)
> COLOUR Char(10)
> With typical data like:
> ROW_ID DEPTH_FROM DEPTH_TO COLOUR
> -------------------
> 1 0 5
> BLUE
> 2 5 8
> BLUE
> 3 8 10
> RED
> 4 10 12
> GREEN
> 5 12 16
> GREEN
> I want to create a view that will 'compress/roll up' the data so
> it appears like:
> DEPTH_FROM DEPTH_TO COLOUR
> ------------------
> 0 8 BLUE
> 8 10 RED
> 10 16 GREEN
> I have been working on this for several days, with no luck,
> any help would be appreciated. BTW: there are no overlaps
> allowed in the depth_from, depth_to values.
> Thanks in advance.

This will also handle gaps between consecutive depth intervals.

CREATE TABLE ColorDepths
(
depth_from INT NOT NULL PRIMARY KEY,
depth_to INT NOT NULL,
color CHAR(10) NOT NULL,
CHECK (depth_from <= depth_to)
)

-- Your sample data augmented to better exercise code
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (0,5, 'BLUE')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (5,8, 'BLUE')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (8,10, 'RED')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (11,12, 'GREEN')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (12,15, 'GREEN')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (16, 18, 'BLUE')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (18, 24, 'BLUE')
INSERT INTO ColorDepths (depth_from, depth_to, color)
VALUES (26, 30, 'BLUE')

-- Associate consecutive depth intervals with natural numbers
CREATE VIEW OrderedColorDepths (depth_from, depth_to, color, seq)
AS
SELECT D1.depth_from, D1.depth_to, D1.color, COUNT(*)
FROM ColorDepths AS D1
INNER JOIN
ColorDepths AS D2
ON D2.depth_from <= D1.depth_from
GROUP BY D1.depth_from, D1.depth_to, D1.color

-- Using above natural numbers, find endpoints
CREATE VIEW ColorDepthEnds (color, seq)
AS
SELECT OD1.color, OD1.seq
FROM OrderedColorDepths AS OD1
LEFT OUTER JOIN
OrderedColorDepths AS OD2
ON OD2.seq = OD1.seq + 1
WHERE OD2.color <> OD1.color OR -- consecutive depths w/ diff. colors
OD2.color IS NULL OR -- last (greatest) depth
OD2.depth_from > OD1.depth_to -- gap between consecutive depths

SELECT color,
MIN(depth_from) AS depth_from , MAX(depth_to) AS depth_to
FROM (SELECT OD.color, OD.depth_from, OD.depth_to,
MIN(DE.seq) AS seq
FROM OrderedColorDepths AS OD
INNER JOIN
ColorDepthEnds AS DE
ON DE.seq >= OD.seq AND
DE.color = OD.color
GROUP BY OD.depth_from, OD.depth_to, OD.color) AS R
GROUP BY seq, color
ORDER BY depth_from

color depth_from depth_to
BLUE 0 8
RED 8 10
GREEN 11 15
BLUE 16 24
BLUE 26 30

Regards,
jag

Representation for Deleted Entities: difficult question

Our customer (of our ecommerce system) wants to be able to preserve
deleted entities in the database so that they can do reporting,
auditing etc.

The system is quite complex where each end user can belong to multiple
institutional affiliations (which can purchase on behalf of the user).
The end user also has a rich trail of past transactions affiliations
etc. Thus in the schema each user entity is related to many others
which in turn relate to yet others and so on.

In the past when a user was deleted all of his complex relationships
were also deleted in a cascading fashion. But now the customer wants
us to add a "deleted" flag to each user so that a user is never
_really_ deleted but instead his "deleted" flag is set to true. The
system subsequently behaves as if the user did not exist but the
customer can still do reports on deleted users.

I pointed out that it is not as simple as that because the user entity
is related to many, many others so we would have to add this "deleted"
flag to every relationship and every other entity and thus have
"deleted" past purchases, "deleted" affiliations - a whole shadow
schema full of such ghost entities. This would overtime degrade
performance since now each query in the system has to add a clause:
"where deleted = 0".

I assume this is a standard problem since many organizations must have
this need of preserving deleted records (for legal or other reasons).
I tried to talk them into creating a simple audit file where all the
deletions will be recorded in XML but they were not too happy with
that.

Is there a more satisfying solution to this than have this "deleted"
flag?

Thanks for your help,

- robertYou didn't post database you're using.

In Oracle you could partition the main table (from where all
'cascaded' is coming from) into deleted/undeleted records. Put a view
on top of the table just as original table would look like with where
clause 'undeleted' and the undeleted partition would always be used.
Only certain versions of oracle allow rows to migrate from partition
to partition, u can look it up.

However, you would still suffer from 'performance' issues on the other
tables, if your data is truly huge.

Best solution would probably be to separate all deleted/undeleted data
into separate tables, and built a union view on top of them for
reporting purposes.

my2c
Someone might have a better idea.

On 13 May 2004 11:32:21 -0700, robertbrown1971@.yahoo.com (Robert
Brown) wrote:

>Our customer (of our ecommerce system) wants to be able to preserve
>deleted entities in the database so that they can do reporting,
>auditing etc.
>The system is quite complex where each end user can belong to multiple
>institutional affiliations (which can purchase on behalf of the user).
>The end user also has a rich trail of past transactions affiliations
>etc. Thus in the schema each user entity is related to many others
>which in turn relate to yet others and so on.
>In the past when a user was deleted all of his complex relationships
>were also deleted in a cascading fashion. But now the customer wants
>us to add a "deleted" flag to each user so that a user is never
>_really_ deleted but instead his "deleted" flag is set to true. The
>system subsequently behaves as if the user did not exist but the
>customer can still do reports on deleted users.
>I pointed out that it is not as simple as that because the user entity
>is related to many, many others so we would have to add this "deleted"
>flag to every relationship and every other entity and thus have
>"deleted" past purchases, "deleted" affiliations - a whole shadow
>schema full of such ghost entities. This would overtime degrade
>performance since now each query in the system has to add a clause:
>"where deleted = 0".
>I assume this is a standard problem since many organizations must have
>this need of preserving deleted records (for legal or other reasons).
>I tried to talk them into creating a simple audit file where all the
>deletions will be recorded in XML but they were not too happy with
>that.
>Is there a more satisfying solution to this than have this "deleted"
>flag?
>Thanks for your help,
>- robert

......
We use Oracle 8.1.7.4 on Solaris 2.7 boxes
remove NSPAM to email|||"Robert Brown" <robertbrown1971@.yahoo.com> wrote in message
news:240a4d09.0405131032.6c2e9802@.posting.google.c om...
> I pointed out that it is not as simple as that because the user entity
> is related to many, many others so we would have to add this "deleted"
> flag to every relationship and every other entity and thus have
> "deleted" past purchases, "deleted" affiliations - a whole shadow
> schema full of such ghost entities. This would overtime degrade
> performance since now each query in the system has to add a clause:
> "where deleted = 0".

Blanket statements like this are rarely true.

> I assume this is a standard problem since many organizations must have
> this need of preserving deleted records (for legal or other reasons).
> I tried to talk them into creating a simple audit file where all the
> deletions will be recorded in XML but they were not too happy with
> that.

So you think file is better than DBMS?

> Is there a more satisfying solution to this than have this "deleted"
> flag?

Can I suggest that there is a modelling problem? I can imagine customer
having multiple timestamp columns, for example

table customer (
...
DOB DATE,
married DATE,
divorced DATE,
died DATE
)

but can't possibly see why you need "is_alive", "is_married" boolean
columns.|||andreyNSPAM@.bookexchange.net (NetComrade) wrote in message news:<40a3cbf2.428366208@.localhost>...

> You didn't post database you're using.

Thanks for your answer. This particular customer is using Oracle but
our software is supported on SQL server as well.

> In Oracle you could partition the main table (from where all
> 'cascaded' is coming from) into deleted/undeleted records. Put a view
> on top of the table just as original table would look like with where
> clause 'undeleted' and the undeleted partition would always be used.
> Only certain versions of oracle allow rows to migrate from partition
> to partition, u can look it up.
> However, you would still suffer from 'performance' issues on the other
> tables, if your data is truly huge.
> Best solution would probably be to separate all deleted/undeleted data
> into separate tables, and built a union view on top of them for
> reporting purposes.
> my2c
> Someone might have a better idea.
>
> On 13 May 2004 11:32:21 -0700, robertbrown1971@.yahoo.com (Robert
> Brown) wrote:
> >Our customer (of our ecommerce system) wants to be able to preserve
> >deleted entities in the database so that they can do reporting,
> >auditing etc.
> >The system is quite complex where each end user can belong to multiple
> >institutional affiliations (which can purchase on behalf of the user).
> >The end user also has a rich trail of past transactions affiliations
> >etc. Thus in the schema each user entity is related to many others
> >which in turn relate to yet others and so on.
> >In the past when a user was deleted all of his complex relationships
> >were also deleted in a cascading fashion. But now the customer wants
> >us to add a "deleted" flag to each user so that a user is never
> >_really_ deleted but instead his "deleted" flag is set to true. The
> >system subsequently behaves as if the user did not exist but the
> >customer can still do reports on deleted users.
> >I pointed out that it is not as simple as that because the user entity
> >is related to many, many others so we would have to add this "deleted"
> >flag to every relationship and every other entity and thus have
> >"deleted" past purchases, "deleted" affiliations - a whole shadow
> >schema full of such ghost entities. This would overtime degrade
> >performance since now each query in the system has to add a clause:
> >"where deleted = 0".
> >I assume this is a standard problem since many organizations must have
> >this need of preserving deleted records (for legal or other reasons).
> >I tried to talk them into creating a simple audit file where all the
> >deletions will be recorded in XML but they were not too happy with
> >that.
> >Is there a more satisfying solution to this than have this "deleted"
> >flag?
> >Thanks for your help,
> >- robert
> ......
> We use Oracle 8.1.7.4 on Solaris 2.7 boxes
> remove NSPAM to email|||Note that SQL Server 2000 EE does support partitioned views.

On 14 May 2004 09:14:47 -0700, robertbrown1971@.yahoo.com (Robert Brown) wrote:

>andreyNSPAM@.bookexchange.net (NetComrade) wrote in message news:<40a3cbf2.428366208@.localhost>...
>> You didn't post database you're using.
>Thanks for your answer. This particular customer is using Oracle but
>our software is supported on SQL server as well.
>
>> In Oracle you could partition the main table (from where all
>> 'cascaded' is coming from) into deleted/undeleted records. Put a view
>> on top of the table just as original table would look like with where
>> clause 'undeleted' and the undeleted partition would always be used.
>> Only certain versions of oracle allow rows to migrate from partition
>> to partition, u can look it up.
>>
>> However, you would still suffer from 'performance' issues on the other
>> tables, if your data is truly huge.
>>
>> Best solution would probably be to separate all deleted/undeleted data
>> into separate tables, and built a union view on top of them for
>> reporting purposes.
>>
>> my2c
>> Someone might have a better idea.
>>
>>
>>
>> On 13 May 2004 11:32:21 -0700, robertbrown1971@.yahoo.com (Robert
>> Brown) wrote:
>>
>> >Our customer (of our ecommerce system) wants to be able to preserve
>> >deleted entities in the database so that they can do reporting,
>> >auditing etc.
>>> >The system is quite complex where each end user can belong to multiple
>> >institutional affiliations (which can purchase on behalf of the user).
>> >The end user also has a rich trail of past transactions affiliations
>> >etc. Thus in the schema each user entity is related to many others
>> >which in turn relate to yet others and so on.
>>> >In the past when a user was deleted all of his complex relationships
>> >were also deleted in a cascading fashion. But now the customer wants
>> >us to add a "deleted" flag to each user so that a user is never
>> >_really_ deleted but instead his "deleted" flag is set to true. The
>> >system subsequently behaves as if the user did not exist but the
>> >customer can still do reports on deleted users.
>>> >I pointed out that it is not as simple as that because the user entity
>> >is related to many, many others so we would have to add this "deleted"
>> >flag to every relationship and every other entity and thus have
>> >"deleted" past purchases, "deleted" affiliations - a whole shadow
>> >schema full of such ghost entities. This would overtime degrade
>> >performance since now each query in the system has to add a clause:
>> >"where deleted = 0".
>>> >I assume this is a standard problem since many organizations must have
>> >this need of preserving deleted records (for legal or other reasons).
>> >I tried to talk them into creating a simple audit file where all the
>> >deletions will be recorded in XML but they were not too happy with
>> >that.
>>> >Is there a more satisfying solution to this than have this "deleted"
>> >flag?
>>> >Thanks for your help,
>>> >- robert
>>
>> ......
>> We use Oracle 8.1.7.4 on Solaris 2.7 boxes
>> remove NSPAM to email|||This solution is not specific to Oracle. Most DBMS support partitions and
views.

--
Mike Nicewarner [TeamSybase]
http://www.datamodel.org
mike@.nospam!datamodel.org
Sybase product enhancement requests:
http://www.isug.com/cgi-bin/ISUG2/submit_enhancement

"NetComrade" <andreyNSPAM@.bookexchange.net> wrote in message
news:40a3cbf2.428366208@.localhost...
> You didn't post database you're using.
> In Oracle you could partition the main table (from where all
> 'cascaded' is coming from) into deleted/undeleted records. Put a view
> on top of the table just as original table would look like with where
> clause 'undeleted' and the undeleted partition would always be used.
> Only certain versions of oracle allow rows to migrate from partition
> to partition, u can look it up.
> However, you would still suffer from 'performance' issues on the other
> tables, if your data is truly huge.
> Best solution would probably be to separate all deleted/undeleted data
> into separate tables, and built a union view on top of them for
> reporting purposes.
> my2c
> Someone might have a better idea.
>
> On 13 May 2004 11:32:21 -0700, robertbrown1971@.yahoo.com (Robert
> Brown) wrote:
> >Our customer (of our ecommerce system) wants to be able to preserve
> >deleted entities in the database so that they can do reporting,
> >auditing etc.
> >The system is quite complex where each end user can belong to multiple
> >institutional affiliations (which can purchase on behalf of the user).
> >The end user also has a rich trail of past transactions affiliations
> >etc. Thus in the schema each user entity is related to many others
> >which in turn relate to yet others and so on.
> >In the past when a user was deleted all of his complex relationships
> >were also deleted in a cascading fashion. But now the customer wants
> >us to add a "deleted" flag to each user so that a user is never
> >_really_ deleted but instead his "deleted" flag is set to true. The
> >system subsequently behaves as if the user did not exist but the
> >customer can still do reports on deleted users.
> >I pointed out that it is not as simple as that because the user entity
> >is related to many, many others so we would have to add this "deleted"
> >flag to every relationship and every other entity and thus have
> >"deleted" past purchases, "deleted" affiliations - a whole shadow
> >schema full of such ghost entities. This would overtime degrade
> >performance since now each query in the system has to add a clause:
> >"where deleted = 0".
> >I assume this is a standard problem since many organizations must have
> >this need of preserving deleted records (for legal or other reasons).
> >I tried to talk them into creating a simple audit file where all the
> >deletions will be recorded in XML but they were not too happy with
> >that.
> >Is there a more satisfying solution to this than have this "deleted"
> >flag?
> >Thanks for your help,
> >- robert
> ......
> We use Oracle 8.1.7.4 on Solaris 2.7 boxes
> remove NSPAM to email|||As Leandro and Mikito point out, you have flaws in your design.
First, deleting the user entity is the only thing that is logically deleted,
but rather than make it an indicator, use a date, as in DELETE_DATE as
nullable. Non-null entities are to be ignored.
In addition, all relationships to the user entity should be evaluated to
determine if they need to be sensitive to the user entity's status. Some
may, and others may not. This is a business question, not a technical
question. For instance, if there are invoices and inventory tables linked
in some way to the user entity, would you really want to *not* display that
information just because an associated user entity had been deleted?

I'd really need to see your design and talk to your business to know exactly
what impact this design change would have on your database.

--
Mike Nicewarner [TeamSybase]
http://www.datamodel.org
mike@.nospam!datamodel.org
Sybase product enhancement requests:
http://www.isug.com/cgi-bin/ISUG2/submit_enhancement

"Robert Brown" <robertbrown1971@.yahoo.com> wrote in message
news:240a4d09.0405131032.6c2e9802@.posting.google.c om...
> Our customer (of our ecommerce system) wants to be able to preserve
> deleted entities in the database so that they can do reporting,
> auditing etc.
> The system is quite complex where each end user can belong to multiple
> institutional affiliations (which can purchase on behalf of the user).
> The end user also has a rich trail of past transactions affiliations
> etc. Thus in the schema each user entity is related to many others
> which in turn relate to yet others and so on.
> In the past when a user was deleted all of his complex relationships
> were also deleted in a cascading fashion. But now the customer wants
> us to add a "deleted" flag to each user so that a user is never
> _really_ deleted but instead his "deleted" flag is set to true. The
> system subsequently behaves as if the user did not exist but the
> customer can still do reports on deleted users.
> I pointed out that it is not as simple as that because the user entity
> is related to many, many others so we would have to add this "deleted"
> flag to every relationship and every other entity and thus have
> "deleted" past purchases, "deleted" affiliations - a whole shadow
> schema full of such ghost entities. This would overtime degrade
> performance since now each query in the system has to add a clause:
> "where deleted = 0".
> I assume this is a standard problem since many organizations must have
> this need of preserving deleted records (for legal or other reasons).
> I tried to talk them into creating a simple audit file where all the
> deletions will be recorded in XML but they were not too happy with
> that.
> Is there a more satisfying solution to this than have this "deleted"
> flag?
> Thanks for your help,
> - robert|||robertbrown1971@.yahoo.com (Robert Brown) wrote:
> Our customer (of our ecommerce system) wants to be able to preserve
> deleted entities in the database so that they can do reporting,
> auditing etc.
> The system is quite complex where each end user can belong to multiple
> institutional affiliations (which can purchase on behalf of the user).
> The end user also has a rich trail of past transactions affiliations
> etc. Thus in the schema each user entity is related to many others
> which in turn relate to yet others and so on.
> In the past when a user was deleted all of his complex relationships
> were also deleted in a cascading fashion.

Users are not deleted. They may die, they may be incarcerated, they may
be fired, but they cannot be deleted. Only data is deleted.

> But now the customer wants
> us to add a "deleted" flag to each user so that a user is never
> _really_ deleted but instead his "deleted" flag is set to true. The
> system subsequently behaves as if the user did not exist but the
> customer can still do reports on deleted users.
> I pointed out that it is not as simple as that because the user entity
> is related to many, many others so we would have to add this "deleted"
> flag to every relationship and every other entity and thus have
> "deleted" past purchases,

If I am hit by a bus tomorrow, will the office furniture I ordered (but
which is owned by the company) disappear? Will the patents I've generated
for the company no longer be valid?

> "deleted" affiliations - a whole shadow
> schema full of such ghost entities.

Why would each of these need it's own deleted flag? If they are being
joined against the user table, then they can just rely on the user table to
provide the necessary logic. If they are not being joined against the user
table, then why would you want them to be excluded based on a condition of
a user?

> This would overtime degrade
> performance since now each query in the system has to add a clause:
> "where deleted = 0".

Do they want the reporting to go back indefinately, or would they like
the data to be "really" deleted after a certain time of "virtual" deletion?

> I assume this is a standard problem since many organizations must have
> this need of preserving deleted records (for legal or other reasons).

It is a standard problem, but there is not a standard solution.

Xho

--
------- http://NewsReader.Com/ -------
Usenet Newsgroup Service $9.95/Month 30GB|||>It is a standard problem, but there is not a standard solution.
>Xho

Put a bit column in there and flag it if this record is "deleted".
Or write a trigger and dump the deleted records into an archive
table.

Randy
http://members.aol.com/rsmeiner|||Robert Brown (robertbrown1971@.yahoo.com) writes:
> Our customer (of our ecommerce system) wants to be able to preserve
> deleted entities in the database so that they can do reporting,
> auditing etc.
> The system is quite complex where each end user can belong to multiple
> institutional affiliations (which can purchase on behalf of the user).
> The end user also has a rich trail of past transactions affiliations
> etc. Thus in the schema each user entity is related to many others
> which in turn relate to yet others and so on.
> In the past when a user was deleted all of his complex relationships
> were also deleted in a cascading fashion. But now the customer wants
> us to add a "deleted" flag to each user so that a user is never
> _really_ deleted but instead his "deleted" flag is set to true. The
> system subsequently behaves as if the user did not exist but the
> customer can still do reports on deleted users.
> I pointed out that it is not as simple as that because the user entity
> is related to many, many others so we would have to add this "deleted"
> flag to every relationship and every other entity and thus have
> "deleted" past purchases, "deleted" affiliations - a whole shadow
> schema full of such ghost entities. This would overtime degrade
> performance since now each query in the system has to add a clause:
> "where deleted = 0".

Indeed, this is quite a big change if this was not in the system from
the beginning.

In our system, many items are not deletable, because even if an item
goes away, there might still be plenty of references to it. So in these
tables - accounts, customers, instruments, currencies, to name a few - there
is a deregdate column. This column serves the double purpose of telling
us if the item is still active, and if it is not, when the entity was
deregistered. And, yes it happens that deregistered entities are revived
too!

A non-trivial issue here is to know when a deregistered item should be
included and when it should not. If you are producing a list of last
month's tranactions, it obviously should. But if you are populating a
list of available products to order, deregistered products should not be
included. So when you introduce this concept in your system, you have
a lot to write in your functional specification too.

Another issue which becomes complicated, is referential integrity.
accounts.accresponsible may refer to the users table, but if the account
is active, the user must be too. Currently we do this in triggers, which
is a bit complex, and more difficult than foreign-key constraints. One
thought I've been playing with is to have tables like active_accounts,
active_users etc. This would not be the partition suggested by others,
since active_accounts would only hold the account number, and foreign
keys to other deregisterable items. Thus, active_accounts.accresponsible
would refer to active_users.userid. The full data would still be in
acconts and users, for both active and deregistered items.

I have never considered the performance cost for "AND deregdate IS NULL",
but I would suggest that if you need to access that column, you probably
already access some column which is not in any index, so there is
already a bookmark lookup, so I would not expect any particular penalty.
(This applies to MS SQL Server. Not that I really expect Oracle to be
different, but I don't know Oracle.)

--
Erland Sommarskog, SQL Server MVP, sommar@.algonet.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp