# pimdav

CalDAV and CardDAV logic without I/O. The caller loads data, calls this
crate, and stores the result. The crate has no database types, no storage
trait and no async code. The server in `server/src/api/pim*.rs` does the
HTTP, authentication, storage and access checks.

The crate uses two libraries:

- **calcard** parses and writes iCalendar and vCard. Its recurrence
  expansion is not used: it drops or duplicates instances on common client
  data. calcard is pinned to one version, because the corpus test reads its
  fixtures.
- **rrule** iterates RRULEs in wall-clock time only. Time zones, UNTIL,
  RDATE, EXDATE and overrides are handled here.

| Module | Content |
|--------|---------|
| `expand` | Recurrence sets |
| `zone` | TZID resolution, VTIMEZONE rules, wall time to UTC |
| `object` | Validation of a PUT body |
| `xml` | WebDAV XML parsing and multistatus building |
| `report` | REPORT request bodies |
| `filter` | `calendar-query` and `addressbook-query` filters |
| `render` | `calendar-data` and `address-data` in responses |
| `freebusy` | Busy time, free-busy replies |
| `itip` | Implicit scheduling as iTIP messages |
| `principal` | Principal search, system address book cards |

The sections below describe how the crate reads the RFCs where the text
leaves room. "We" means this crate and the server that uses it.

## Recurrence expansion

`expand::expand` returns the instances that overlap a time window. A
zero-length instance overlaps if it starts inside the window.

### The recurrence set

- The set is DTSTART, plus RRULE and RDATE, minus EXRULE and EXDATE.
  Duplicates count once.
- DTSTART is always an instance, even when it does not match the RRULE.
  The exception: every RRULE has an UNTIL before DTSTART. Then the series
  has no instances from its rules and no DTSTART.
- A UTC UNTIL is converted to wall time in the zone of DTSTART before the
  comparison. The rule iterates in wall time. Without the conversion, a
  series east of UTC loses its last instance.
- A DATE UNTIL on a DATE-TIME series includes that whole day. A floating
  UNTIL on a zoned series counts as wall time.
- COUNT and UNTIL together are invalid. We ignore COUNT.
- EXRULE (RFC 2445) is applied.
- Several RRULEs are united.
- Components are grouped by component type and UID. Each group is one
  master with its overrides. A component without UID stands alone.
- A component with RECURRENCE-ID is an override, even if it carries an
  RRULE.
- Several masters for one UID are invalid. The highest SEQUENCE wins, and
  the first one wins a tie.
- A component without DTSTART is skipped.

### Overrides

- An override matches its instance by RECURRENCE-ID only.
- SEQUENCE never decides whether an override applies. It only picks
  between two overrides of the same instance. The first one wins a tie.
- An EXDATE also removes an override of that instance.
- An override whose RECURRENCE-ID matches no instance still shows. Clients
  display it, so hiding it would lose data.
- THISANDFUTURE moves every later instance by the same offset and gives it
  the override's length. The latest such override before an instance
  applies. A later plain override still matches by its original time.
- An all-day series matches RECURRENCE-ID and EXDATE by date alone,
  whatever zone the value names.

### Values and zones

- A DATE value ignores its TZID. RFC 5545 forbids the combination.
- A floating RECURRENCE-ID, RDATE or EXDATE on a zoned series is read in
  the series zone. Clients write them that way.
- A DATE EXDATE on a DATE-TIME series removes every instance on that day.
- A TZID on a UTC value is ignored. The `Z` wins.
- The caller passes a zone for floating values and dates. The server uses
  the query's `timezone`, then the collection's `calendar-timezone`, then
  UTC.

### Durations and DST

- DTEND gives an exact length. Every instance keeps it, so a series across
  a DST change keeps 23 hours if the first instance had 23.
- DURATION gives a nominal length. Whole days count in wall time, so a day
  can last 23 or 25 hours.
- Without DTEND or DURATION, a date lasts one day and a date-time lasts
  zero seconds.
- DTEND before DTSTART gives a length of zero.
- A wall time in a DST gap takes the offset before the gap. So 02:30 on a
  spring-forward day becomes 03:30 (RFC 5545, 3.3.5).
- A repeated wall time takes its first occurrence.
- Two instances that land on the same instant after the gap rule merge
  into one. They would have the same RECURRENCE-ID.

### Time zones

A TZID is resolved in this order:

1. An exact IANA name. It wins over the VTIMEZONE rules. Clients send
   stale rules, and the user means the place.
2. A guessed IANA name. Sources are vendor prefixes such as
   `/mozilla.org/.../Europe/Berlin`, calcard's Windows name table,
   `X-LIC-LOCATION` and `X-MICROSOFT-CDO-TZID`.
3. The VTIMEZONE rules, evaluated from STANDARD and DAYLIGHT.
4. A fixed offset from a label such as `(UTC+02:00) Athens`.
5. The floating zone.

A guess (step 2) must agree with the VTIMEZONE rules. The check compares
the offsets in mid-January and mid-July of 2020 to 2022. On disagreement
the rules win. This caught calcard mapping `(GMT+01.00) Sarajevo` to
Lisbon.

A repeated TZID in one object is invalid. The first definition wins.

Limits of the VTIMEZONE evaluation:

- Rules repeat their last offset after the year 2200.
- One observance yields at most 5,000 onsets.
- The wall-time conversion assumes at most one offset change within a
  day.

### Rules we do not expand

These rules yield no instances. DTSTART stays an instance. RFC 7529 allows
treating an unsupported rule as absent.

- `RSCALE` other than `GREGORIAN`.
- `SKIP` other than `OMIT`.
- A leap month such as `BYMONTH=5L`.
- A BYMONTH value outside 1 to 12.
- An UNTIL that does not parse.

### The rrule crate workaround

rrule treats `BYDAY=1MO,TU` as an intersection of the plain and numbered
weekdays. RFC 5545 means a union. When a rule mixes both forms, we number
every occurrence of the plain weekdays. For a MONTHLY rule, or a YEARLY
rule with BYMONTH, that is `1TU` to `5TU`. Otherwise it is `1TU` to `53TU`.

### Cost limits

- One rule generates at most 1,000,000 occurrences per expansion. Past
  that, `Expansion::truncated` is set and later instances are missing.
  Iteration always starts at DTSTART, so a `FREQ=MINUTELY` rule from years
  ago reaches the cap.
- A truncated expansion counts as matching in a time-range filter. A false
  match costs the client one extra object. A false miss would hide an
  event.

## Reports and filters

### Validation of a PUT body

- A calendar object has one UID, one component type and no METHOD.
  Otherwise it fails `valid-calendar-object-resource`.
- A vCard without UID is accepted. Several clients omit it. The server
  uses the resource name instead.

### Text matching

| Collation | Behavior |
|-----------|----------|
| `i;octet` | Exact bytes |
| `i;ascii-casemap` | ASCII letters fold. Default for CalDAV. |
| `i;unicode-casemap` | NFKD, then lowercase. Default for CardDAV. |

Any other collation fails with `supported-collation`.

RFC 5051 folds with titlecase mappings. We lowercase. For matching, both
give the same result except for a few digraph characters.

Text matching reads a property through calcard's writer. There is one data
model, so component indices match the expansion.

### Time ranges

- The rules of RFC 4791, 9.9 apply to VEVENT, VTODO, VJOURNAL, VFREEBUSY
  and VALARM.
- VTODO follows the full table, including todos without DTSTART.
- A VALARM trigger fires once per instance of its parent. RELATED=END and
  REPEAT count.

### Errors

- A filter that does not parse fails with `CALDAV:valid-filter` or
  `CARDDAV:valid-filter`.
- Another malformed REPORT body answers 400.

### `calendar-data` and `address-data`

- The stored bytes come back unchanged unless the request narrows,
  expands or converts them.
- `expand` returns one component per instance in UTC with a
  RECURRENCE-ID and without RRULE, RDATE or EXDATE. Dates stay dates.
- One object may expand into at most 10,000 instances. Past that the
  REPORT fails with `CALDAV:max-instances`.
- `address-data` can convert between vCard 3.0 and 4.0. calcard then
  writes `CHARSET=UTF-8` on non-ASCII 3.0 values.
- CR is written as `
` in XML, so returned data keeps its CRLF.

### `max-instances`

We do not advertise `max-instances`, and PUT never checks it. An endless
RRULE is valid and common. The occurrence cap protects the CPU instead.

### Sync

- A token is `urn:fbng:sync:{collection id}-{seq}`. The id makes a token
  of a deleted collection invalid for a new one at the same URL.
- A token of another collection, or one newer than the collection, fails
  with `valid-sync-token`.
- Deleted members come back as 404 responses.
- With a limit, the oldest changes come first. The response adds a 507
  for the collection and hands out the token of its last change.
- The system address book has no change log. Only its current token is
  valid. After any principal change, clients get `valid-sync-token` and
  sync from scratch.

### `addressbook-query`

- `test` defaults to `anyof`, on the filter and on each prop-filter.
- `nresults` truncates the result and adds a 507 for the collection.

### Principal search

- `principal-property-search` and Apple's
  `calendarserver-principal-search` search the display name, the calendar
  user addresses and the calendar user type.
- A search without terms lists every principal. python-caldav sends such
  searches.

## Scheduling

`itip` implements implicit scheduling (RFC 6638) between principals of one
server. The caller maps addresses onto its principals through closures.

### What sends messages

- An organizer PUT sends a REQUEST to each attendee. Each attendee only
  gets the instances they are part of.
- An attendee left out of an override gets an EXDATE for that instance.
- Removed attendees get a CANCEL. An organizer DELETE cancels for
  everyone.
- A change of only DTSTAMP, LAST-MODIFIED, CREATED, SEQUENCE, SENT-BY or
  X- properties sends nothing.
- `SCHEDULE-AGENT=CLIENT` or `NONE` sends nothing for that address. An
  unknown value counts as `NONE`.
- `SCHEDULE-FORCE-SEND=REQUEST` sends a REQUEST even without a change.
- An attendee's change of their own PARTSTAT sends a REPLY. This works per
  instance, through an override or an added EXDATE.
- An attendee DELETE declines every instance, unless the request carries
  `Schedule-Reply: F`.

### What each side may change

- The server owns the PARTSTAT of every attendee in the organizer object.
  An organizer PUT cannot set them.
- A reschedule resets all attendees to NEEDS-ACTION and raises SEQUENCE.
  Shortening a series or adding an EXDATE is no reschedule.
- An attendee may not change times, rules, the ORGANIZER or the attendee
  list. That fails with `allowed-attendee-scheduling-object-change`. An
  attendee may add EXDATEs, which declines those instances.
- Other attendee changes, such as SUMMARY or alarms, stay in the
  attendee's copy.

### Delivery into copies

- A new copy goes into the attendee's default calendar. That is their
  oldest calendar that takes VEVENT.
- An update keeps the attendee's alarms, TRANSP, PERCENT-COMPLETE and
  COMPLETED.
- A CANCEL sets STATUS:CANCELLED on the copy. Nothing is deleted.
- An attendee who deleted their copy gets it back with the next organizer
  update that is not quiet.
- A REPLY for one instance creates an override in the organizer object.
- An update that only changes other attendees' answers is quiet. It keeps
  the copy's Schedule-Tag and leaves no inbox entry.

### Status codes

The organizer object carries the status on each ATTENDEE. An attendee's
copy carries the status of its last reply on the ORGANIZER.

| SCHEDULE-STATUS | Meaning |
|-----------------|---------|
| 1.2 | Delivered to a local principal |
| 2.0 | The attendee replied. The reply's REQUEST-STATUS wins if present. |
| 3.7 | An address in our domains that names no one, or a disabled account |
| 5.2 | An external address. There is no iMIP. |
| 5.3 | The recipient has no calendar for the component, for example a VTODO to a room |

### Schedule-Tag

- The tag is the ETag of the version that last changed it.
- A direct PUT by the attendee changes the tag, as RFC 6638, 3.2.10
  requires.
- A server-side merge of other attendees' answers keeps it.
- caldav-server-tester flags `schedule-tag.stable-partstat`. It expects
  the attendee's own PARTSTAT PUT to keep the tag. We follow the RFC.

### SENT-BY

- The server owns SENT-BY. Every write of a scheduling object sets it to
  the writer, or removes it when the owner writes.
- It goes on the owner's ORGANIZER for invitations and on the owner's
  ATTENDEE for replies.

### Free-busy

- Free-busy covers the principal's own calendars that take VEVENT and are
  opaque (`schedule-calendar-transp`). It never covers the inbox or lent
  calendars.
- Transparent and cancelled events are free. Stored VFREEBUSY components
  count with their FBTYPE.
- For scheduling (outbox POST and room conflicts), the principal's own
  answer counts too. A declined instance is free. NEEDS-ACTION and
  TENTATIVE count as BUSY-TENTATIVE. RFC 6638 does not define this rule.
- The `free-busy-query` REPORT follows RFC 4791 and ignores PARTSTAT.

### Rooms and resources

- The server answers an invitation for a room before messages go out.
- An instance that overlaps an existing booking is declined. A declined
  instance of a series gets its own override. Everything else is
  accepted.
- Tentative bookings block. Cancelled bookings, declined bookings and
  copies of the same UID do not.
- Only instances in the next 366 days are checked. Past instances and
  later ones are accepted unchecked.

## Where the RFCs are unclear or implementations differ

| Case | What we do | Why |
|------|------------|-----|
| DTSTART does not match the RRULE | DTSTART is an instance | RFC 5545 calls the set undefined; libical, Google and Apple include it |
| TZID on a DATE value | Ignore the TZID | RFC 5545 forbids applying it |
| TZID on a UTC value | Use the `Z` | RFC 5545 forbids the TZID there |
| DTEND before DTSTART | Length zero | Other readings invent a start time |
| Orphan override | Show it | Clients show it; hiding loses the user's edit |
| EXDATE and an override on one instance | Remove both | The RFC excludes the instance; a stale override stays after a delete |
| COUNT and UNTIL together | Use UNTIL | Invalid input; UNTIL is the safer bound |
| Several masters per UID | Highest SEQUENCE, first on a tie | Invalid input; matches recurring-ical-events |
| Wall times in a DST gap that collide | Merge them | They share one RECURRENCE-ID |
| Floating EXDATE or RECURRENCE-ID on a zoned series | Read in the series zone | Thunderbird writes them that way |
| Missing `Depth` on PROPFIND | Depth 0 | RFC 4918 says infinity; clients that omit it mean 0 |
| `Depth` on REPORT | Ignored | Queries always cover the collection's members |
| Organizer sets another attendee's PARTSTAT | Overridden, not rejected | The RFC says SHOULD reject; overriding keeps clients working |
| Organizer resets an attendee to NEEDS-ACTION | Not possible | The RFC's MAY; `SCHEDULE-FORCE-SEND` invites again |
| CANCEL to an attendee | STATUS:CANCELLED on the copy | Matches sabre/dav; the attendee sees what happened |
| Attendee PARTSTAT PUT and Schedule-Tag | The tag changes | RFC 6638, 3.2.10; caldav-server-tester expects the opposite |
| `schedule-send*` privileges on a shared calendar | Listed there | RFC 6638 puts them on the outbox, which a sharee cannot see |
| PARTSTAT in scheduling free-busy | Own answer counts | Not defined by RFC 6638 |
| `/.well-known` redirect | 307 | A 301 drops the REPORT body; RFC 6764 allows 307 |

## Not handled

- **THISANDFUTURE in scheduling.** It is treated as a plain override.
  Clients split a series instead: they end it with UNTIL and create a new
  UID. Expansion does handle THISANDFUTURE.
- **Delegation** (`DELEGATED-TO`). The attendee's change is rejected,
  because attendees may not change the attendee list.
- **Task progress replies.** An assignee's PERCENT-COMPLETE and COMPLETED
  stay in their copy.
- **Non-Gregorian calendars** (RSCALE). See "Rules we do not expand".
- **Full RFC 5051 folding.** We lowercase after NFKD instead of using
  titlecase mappings.
- **iMIP.** No email is sent or read. External attendees get 5.2.
- **DTSTAMP on organizer objects.** Objects are stored as sent, even
  without DTSTAMP.
- **Reports on a single object or a home.** Only collections take
  calendar and address book reports. Others fail with `supported-report`.

## Testing

```bash
cargo test -p pimdav
```

| File | Covers |
|------|--------|
| `tests/expand.rs` | Recurrence cases the corpus cannot check |
| `tests/corpus.rs` | Differential test of `expand` over 998 real files |
| `tests/protocol.rs` | XML parsing and building, PUT validation |
| `tests/report.rs` | Filters, rendering, free-busy |
| `tests/itip.rs` | Scheduling messages and copies |
| `tests/principal.rs` | Principal search, system address book cards |

The server's end-to-end tests are `server/tests/api_pim*.rs`.

### The corpus test

The test expands every `.ics` fixture of calcard and compares the start
times with the Python library recurring-ical-events.

- The fixtures come from calcard's crate sources in the cargo registry,
  under `$CARGO_HOME` or `~/.cargo`. The pinned calcard version in
  `Cargo.toml` and the `CALCARD` constant in the test must match.
- Both sides expand VEVENTs with starts in 1970 to 2040. Floating times
  and dates count as UTC. X-WR-TIMEZONE is ignored.
- `tests/corpus/oracle.tsv` holds the expected count and a hash per file.
  `oracle.py` writes it.
- `tests/corpus/deviations.tsv` overrides the oracle for a file. It
  covers files the oracle cannot parse and files where we differ on
  purpose.

To regenerate the oracle, for example after a calcard update:

```bash
cd pimdav/tests/corpus
F=$(ls -d ~/.cargo/registry/src/*/calcard-<version>/resources/ical)
uv run oracle.py "$F" > oracle.tsv
uv run oracle.py "$F" --full > /tmp/oracle-full.tsv
```

To review a difference, write our full output and compare it with the
oracle's:

```bash
PIMDAV_CORPUS_FULL=/tmp/ours-full.tsv cargo test -p pimdav --test corpus
```

On failure, the test prints one line per differing file in the format of
`deviations.tsv`. For each file, fix the code or add the line with a
reason. Every line in `deviations.tsv` needs a reason. A line without one
hides a bug.