CalDAV/CardDAV performance quick wins
- Generated collections: CTag and sync token from their sources, members built once per REPORT; birthday sync at 10k contacts from ~11 min to 0.2 s - CPU work of REPORTs, large PROPFINDs, feeds, the JSON views and busy time runs on the blocking pool, so heavy requests no longer stall the app - One REPORT answer expands at most 20,000 instances, then ends with a 507 - Incremental sync query split so SQLite uses the seq index - JSON event detail loads one object, not the whole collection - Default collections remembered per principal instead of three writes per DAV request; hot user lookups use cached statements - jemalloc as the allocator: idle RSS after load from 2.2 GB to about 50 MB; the build images install make for it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
M.hearthforge-ci.toml
@@ -55,8 +55,9 @@ name = "setup"
timeout = 900
run_sh = """
# openssl-dev + openssl-libs-static: webauthn-rs links OpenSSL, and this is
# a static musl build. Mirrors the Containerfile's build stage.
apk add --no-cache musl-dev binaryen just curl libstdc++ podman-remote \
# a static musl build. make builds jemalloc. Mirrors the Containerfile's
# build stage.
apk add --no-cache musl-dev binaryen just curl libstdc++ podman-remote make \
openssl-dev openssl-libs-static pkgconfig
# The official rust images use rustup's minimal profile, so rustfmt and
MCargo.lock
@@ -2919,6 +2919,7 @@ dependencies = [
"sha2 0.11.0",
"tar",
"tempfile",
"tikv-jemallocator",
"tokio",
"tower",
"tower-http",
@@ -3326,6 +3327,26 @@ dependencies = [
"zune-jpeg",
]
[[package]]
name = "tikv-jemalloc-sys"
version = "0.6.1+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd8aa5b2ab86a2cefa406d889139c162cbb230092f7d1d7cbc1716405d852a3b"
dependencies = [
"cc",
"libc",
]
[[package]]
name = "tikv-jemallocator"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0359b4327f954e0567e69fb191cf1436617748813819c94b8cd4a431422d053a"
dependencies = [
"libc",
"tikv-jemalloc-sys",
]
[[package]]
name = "time"
version = "0.3.55"
MContainerfile
@@ -13,7 +13,8 @@ ARG TRUNK_VERSION
# binaryen so trunk uses the system wasm-opt instead of downloading a glibc binary that cannot run on musl
# openssl-dev + openssl-libs-static for webauthn-rs, whose core crate links
# OpenSSL. The binary is static, so nothing is needed in the runtime image.
RUN apk add --no-cache curl ca-certificates musl-dev binaryen just \
# make builds jemalloc, the server's allocator.
RUN apk add --no-cache curl ca-certificates musl-dev binaryen just make \
openssl-dev openssl-libs-static pkgconfig \
&& rustup target add wasm32-unknown-unknown
MREADME.md
@@ -490,7 +490,8 @@ for the old one.
- An inbox keeps its newest 100 messages. The meetings themselves stay in
the calendar.
- A repeating event returned as single instances can have at most 10,000
in one request.
in one request, and one request returns at most 20,000 such instances.
Past that, apps get a shortened answer.
- A repeating rule is followed for at most 1,000,000 occurrences per
request. Only extreme rules reach this, such as every minute for years.
Such an event counts as matching every time range.
@@ -500,9 +501,9 @@ for the old one.
- One lock serializes all writes of calendar entries and contacts on the
server. That is fine for a small server.
- The system address book and the birthday calendar have no change
history. After any change to accounts or rooms, or to a birthday, apps
download them again in full. The birthday calendar is rebuilt from all
your contacts on each request.
history. After any change to accounts or rooms, or to any of your
contacts, apps download them again in full. The birthday calendar is
rebuilt from all your contacts when an app reads it.
Not supported:
Mpimdav/README.md
@@ -212,6 +212,11 @@ model, so component indices match the expansion.
RECURRENCE-ID and without RRULE, RDATE or EXDATE. Dates stay dates.
- One object may expand into at most 10,000 instances. Past that the
REPORT fails with `CALDAV:max-instances`.
- One answer may expand into at most 20,000 instances across its objects.
Past that it stops before the next object and ends with a 507 for the
collection, with `DAV:number-of-matches-within-limits`, as for a client
limit. RFC 4791 (7.8) lets a server cut results short. This bounds the
memory of one request: a year of a 50,000-event calendar was 247 MB.
- `address-data` without a version means 3.0 (RFC 6352, 10.4). A 4.0
card is then converted, see "vCard versions" below. calcard writes
`CHARSET=UTF-8` on non-ASCII 3.0 values.
@@ -231,9 +236,10 @@ RRULE is valid and common. The occurrence cap protects the CPU instead.
- Deleted members come back as 404 responses.
- With a limit, the oldest changes come first. The response adds a 507
for the collection and hands out the token of its last change.
- The system address book has no change log. Only its current token is
valid. After any principal change, clients get `valid-sync-token` and
sync from scratch.
- The system address book has no change log. Its token is a hash of the
principal list, so it is known without building the cards. Only the
current token is valid. After any principal change, clients get
`valid-sync-token` and sync from scratch.
### `addressbook-query`
@@ -453,9 +459,11 @@ the output unchanged. Only line endings become CRLF.
language.
The server builds the birthday calendar of a principal from its own
address books only, on each request. Like the system address book, it has
no change log: its sync token is a hash of its members' ETags, and only
the current token is valid. It is read-only, not shareable, has no feed
address books only, when a client reads its members. Like the system
address book, it has no change log: its sync token is a hash of the
address books' ids and change counters, and only the current token is
valid. So any contact change, not only a birthday, makes clients resync
it. A REPORT that reads many members builds them once. It is read-only, not shareable, has no feed
links, and is left out of free-busy and of the choice of the calendar that
receives invitations.
Mpimdav/src/render.rs
@@ -136,21 +136,23 @@ pub fn address_request(e: &Element) -> Result<AddressData, Refused> {
})
}
/// The calendar-data of one object. The stored text is returned unchanged
/// unless the request narrows or expands it.
/// The calendar-data of one object, and how many instances an `expand`
/// produced. The stored text is returned unchanged unless the request
/// narrows or expands it.
pub fn calendar_data(
raw: &str,
req: &CalendarData,
floating: &Zone,
) -> Result<String, TooManyInstances> {
) -> Result<(String, usize), TooManyInstances> {
if *req == CalendarData::default() {
return Ok(raw.to_string());
return Ok((raw.to_string(), 0));
}
let Ok(mut cal) = ICalendar::parse(raw) else {
return Ok(raw.to_string());
return Ok((raw.to_string(), 0));
};
let mut instances = 0;
if let Some(r) = &req.expand {
cal = expanded(&cal, r, floating)?;
(cal, instances) = expanded(&cal, r, floating)?;
}
if let Some(r) = &req.limit_recurrence {
cal = limit_recurrence(&cal, r, floating);
@@ -163,7 +165,7 @@ pub fn calendar_data(
copy(&cal, 0, Some(sel), &mut out);
cal = ICalendar { components: out };
}
Ok(cal.to_string())
Ok((cal.to_string(), instances))
}
/// One component per instance in `range`, in UTC, with a RECURRENCE-ID and
@@ -172,9 +174,10 @@ fn expanded(
cal: &ICalendar,
range: &TimeRange,
floating: &Zone,
) -> Result<ICalendar, TooManyInstances> {
) -> Result<(ICalendar, usize), TooManyInstances> {
let exp = expand(cal, range.clone(), floating.clone());
if exp.truncated || exp.instances.len() > MAX_EXPANDED {
let count = exp.instances.len();
if exp.truncated || count > MAX_EXPANDED {
return Err(TooManyInstances);
}
let mut out = vec![root(cal)];
@@ -228,7 +231,7 @@ fn expanded(
}
out[0].component_ids.push(at);
}
Ok(ICalendar { components: out })
Ok((ICalendar { components: out }, count))
}
/// The masters, and only the overrides that affect `range` (RFC 4791,
Mpimdav/tests/report.rs
@@ -172,7 +172,8 @@ fn expand_keeps_dates_and_limit_keeps_used_overrides() {
&req(r#"<c:expand start="20260102T000000Z" end="20260104T000000Z"/>"#),
&Zone::Utc,
)
.unwrap();
.unwrap()
.0;
assert!(out.contains("DTSTART;VALUE=DATE:20260102"), "{out}");
assert!(out.contains("RECURRENCE-ID;VALUE=DATE:20260103"), "{out}");
assert!(!out.contains("RRULE") && !out.contains("20260104"), "{out}");
@@ -182,7 +183,8 @@ fn expand_keeps_dates_and_limit_keeps_used_overrides() {
&req(r#"<c:limit-recurrence-set start="20260102T000000Z" end="20260103T000000Z"/>"#),
&Zone::Utc,
)
.unwrap();
.unwrap()
.0;
assert!(
out.contains("RRULE") && out.contains("moved") && !out.contains("later"),
"{out}"
Mserver/Cargo.toml
@@ -12,6 +12,10 @@ path = "src/main.rs"
api-types = { path = "../api-types" }
pimdav = { path = "../pimdav" }
anyhow = "1"
# Global allocator, for C code too. glibc keeps freed memory after a burst of
# large answers, musl's allocator (the static release) is slow, and mimalloc
# kept hundreds of MB after load. Needs `make` to build.
tikv-jemallocator = { version = "0.6", features = ["unprefixed_malloc_on_supported_platforms", "background_threads"] }
argon2 = "0.6"
axum = "0.8"
bytes = "1"
Mserver/src/api/pim.rs
@@ -39,6 +39,7 @@ use pimdav::xml::{
use pimdav::zone::{self, Zone};
use pimdav::{contact, filter, freebusy, object};
use super::common::blocking;
use super::pim_schedule::{self, Directory, Stored, Writer};
use sha2::{Digest, Sha256};
use xmltree::Element;
@@ -102,6 +103,10 @@ const LOCAL_NO_DOT: &AsciiSet = &LOCAL.add(b'.');
type Reply = Result<Response<Body>, ApiError>;
/// Up to this many responses a PROPFIND answer is built in place. Larger ones
/// go to the blocking pool, so they do not stall the async workers.
const INLINE_RESPONSES: usize = 64;
/// `{WELL_KNOWN_CALDAV}` and `{WELL_KNOWN_CARDDAV}`.
///
/// 307, not 301: HTTP clients drop the body on a 301, and python-caldav sends
@@ -133,6 +138,7 @@ pub async fn handle(State(state): State<Arc<AppState>>, req: Request<Body>) -> R
}
/// The signed-in account.
#[derive(Clone)]
struct Me {
id: i64,
/// The account's principal, which owns its collections.
@@ -148,6 +154,7 @@ struct Me {
/// The principal whose URLs a request addresses: the signed-in account, or
/// a room or resource. Another account's principal is readable too.
#[derive(Clone)]
struct Space {
id: i64,
/// The URL segment, as the request spelled it.
@@ -516,22 +523,19 @@ pub(super) fn generated(id: i64) -> bool {
id <= DIRECTORY
}
/// A generated collection. Its members' ETags stand in for a change counter:
/// any change to them changes the CTag and the sync token. Only the current
/// token is valid, so a client resyncs after each change.
/// A generated collection. Its CTag and sync token come from `source`, what
/// its members are built from, so they are known without building them.
/// Only the current token is valid, so a client resyncs after each change.
fn generated_collection(
id: i64,
slug: &str,
name: &str,
components: &str,
members: &[(PimObject, Vec<u8>)],
source: &str,
) -> PimCollection {
let digest = Sha256::digest(
members
.iter()
.map(|(o, _)| o.etag.as_str())
.collect::<String>(),
);
// Bump when the members built from the same source change.
const FORMAT: &str = "1";
let digest = Sha256::digest(format!("{FORMAT}\n{source}"));
PimCollection {
id,
slug: slug.to_string(),
@@ -543,9 +547,28 @@ fn generated_collection(
}
pub(super) type Members = Vec<(PimObject, Vec<u8>)>;
type MemberMap = std::collections::HashMap<String, (PimObject, Vec<u8>)>;
/// The generated system address book: one card per visible principal.
pub(super) async fn directory(state: &AppState) -> Result<(PimCollection, Members), ApiError> {
/// The generated system address book.
pub(super) async fn directory_collection(state: &AppState) -> Result<PimCollection, ApiError> {
let source: String = state
.db
.pim_principals()
.await?
.iter()
.map(|p| format!("{}\t{}\t{}\t{:?}\n", p.id, p.name, p.display(), p.kind))
.collect();
Ok(generated_collection(
DIRECTORY,
DIRECTORY_SLUG,
"Directory",
"",
&source,
))
}
/// The members of the system address book: one card per visible principal.
pub(super) async fn directory(state: &AppState) -> Result<Members, ApiError> {
let mut members = Vec::new();
for p in state.db.pim_principals().await? {
let uuid = principal_uuid(p.id);
@@ -563,38 +586,57 @@ pub(super) async fn directory(state: &AppState) -> Result<(PimCollection, Member
data,
));
}
let col = generated_collection(DIRECTORY, DIRECTORY_SLUG, "Directory", "", &members);
Ok((col, members))
Ok(members)
}
/// The generated birthday calendar of a principal: the birthdays and
/// anniversaries in its own address books, not lent ones.
// ponytail: rebuilt from every contact on each request. Store the events if
// large address books make it slow.
pub(super) async fn birthdays(
/// The generated birthday calendar of a principal. It changes whenever one
/// of the principal's own address books does.
pub(super) async fn birthdays_collection(
state: &AppState,
principal: i64,
) -> Result<(PimCollection, Members), ApiError> {
let mut members = Vec::new();
) -> Result<PimCollection, ApiError> {
let source: String = state
.db
.pim_collections(principal, PimKind::AddressBook)
.await?
.iter()
.map(|b| format!("{}:{}\n", b.id, b.seq))
.collect();
let mut col = generated_collection(BIRTHDAYS, BIRTHDAYS_SLUG, "Birthdays", "VEVENT", &source);
col.transparent = true;
Ok(col)
}
/// The members of the birthday calendar: the birthdays and anniversaries in
/// the principal's own address books, not lent ones.
// ponytail: rebuilt from every contact on each request. Store the events if
// large address books make it slow.
pub(super) async fn birthdays(state: &AppState, principal: i64) -> Result<Members, ApiError> {
let mut books = Vec::new();
for book in state
.db
.pim_collections(principal, PimKind::AddressBook)
.await?
{
for (o, data) in state.db.pim_objects_with_data(book.id).await? {
let key = format!("{}/{}", book.id, o.name);
for (uid, ics) in contact::dates(&String::from_utf8_lossy(&data), &key) {
let data = ics.into_bytes();
members.push((
generated_object(format!("{uid}.ics"), uid, "VEVENT", &data),
data,
));
books.push((book.id, state.db.pim_objects_with_data(book.id).await?));
}
blocking(move || -> Result<Members, ApiError> {
let mut members = Vec::new();
for (book, objects) in books {
for (o, data) in objects {
let key = format!("{book}/{}", o.name);
for (uid, ics) in contact::dates(&String::from_utf8_lossy(&data), &key) {
let data = ics.into_bytes();
members.push((
generated_object(format!("{uid}.ics"), uid, "VEVENT", &data),
data,
));
}
}
}
}
let mut col = generated_collection(BIRTHDAYS, BIRTHDAYS_SLUG, "Birthdays", "VEVENT", &members);
col.transparent = true;
Ok((col, members))
Ok(members)
})
.await
}
/// The members of collection `id`, stored or generated. `principal` owns
@@ -605,8 +647,8 @@ pub(super) async fn members_of(
id: i64,
) -> Result<Members, ApiError> {
match id {
DIRECTORY => Ok(directory(state).await?.1),
BIRTHDAYS => Ok(birthdays(state, principal).await?.1),
DIRECTORY => directory(state).await,
BIRTHDAYS => birthdays(state, principal).await,
id => Ok(state.db.pim_objects_with_data(id).await?),
}
}
@@ -664,8 +706,10 @@ impl Cx<'_> {
}));
}
let generated = match (kind, slug) {
(PimKind::AddressBook, DIRECTORY_SLUG) => Some(directory(self.state).await?.0),
(PimKind::Calendar, BIRTHDAYS_SLUG) => Some(birthdays(self.state, space.id).await?.0),
(PimKind::AddressBook, DIRECTORY_SLUG) => Some(directory_collection(self.state).await?),
(PimKind::Calendar, BIRTHDAYS_SLUG) => {
Some(birthdays_collection(self.state, space.id).await?)
}
_ => None,
};
if let Some(c) = generated {
@@ -711,8 +755,8 @@ impl Cx<'_> {
.collect();
if space.mine {
let generated = match kind {
PimKind::AddressBook => directory(self.state).await?.0,
PimKind::Calendar => birthdays(self.state, space.id).await?.0,
PimKind::AddressBook => directory_collection(self.state).await?,
PimKind::Calendar => birthdays_collection(self.state, space.id).await?,
};
out.push(Col {
c: generated,
@@ -730,6 +774,24 @@ impl Cx<'_> {
members_of(self.state, self.space().id, c.id).await
}
async fn member_map(&self, c: &PimCollection) -> Result<MemberMap, ApiError> {
Ok(self
.members(c)
.await?
.into_iter()
.map(|m| (m.0.name.clone(), m))
.collect())
}
/// A generated collection is built as a whole, so a REPORT that looks up
/// many of its members builds it once.
async fn generated_members(&self, c: &PimCollection) -> Result<Option<MemberMap>, ApiError> {
match generated(c.id) {
true => Ok(Some(self.member_map(c).await?)),
false => Ok(None),
}
}
async fn member(
&self,
c: &PimCollection,
@@ -924,13 +986,29 @@ impl Cx<'_> {
}
}
let mut responses = Vec::with_capacity(list.len());
let described_len = list.len();
let mut described = Vec::with_capacity(described_len);
for (href, res) in list {
let mut all = self.props(&res);
all.extend(self.dead_props(&res).await?);
responses.push(select(href, &request, all));
let dead = self.dead_props(&res).await?;
described.push((href, res, dead));
}
Ok(multistatus(&responses, None))
let answer = move |me: &Me, space: Option<&Space>| {
let responses: Vec<_> = described
.into_iter()
.map(|(href, res, dead)| {
let mut all = live_props(me, space, &res);
all.extend(dead);
select(href, &request, all)
})
.collect();
multistatus(&responses, None)
};
// A handoff to the blocking pool costs more than a small answer.
if described_len <= INLINE_RESPONSES {
return Ok(answer(self.me, self.space));
}
let (me, space) = (self.me.clone(), self.space.cloned());
blocking(move || -> Reply { Ok(answer(&me, space.as_ref())) }).await
}
/// The client properties stored for a resource.
@@ -953,226 +1031,226 @@ impl Cx<'_> {
.collect())
}
/// Every live property of a resource, with its value.
fn props(&self, res: &Res) -> Vec<Element> {
let text = |ns: &str, local: &str, v: &str| with_text(el(ns, local), v);
let href_prop = |ns: &str, local: &str, h: &str| with_children(el(ns, local), hrefs([h]));
let resourcetype = |types: &[(&str, &str)]| {
with_children(
el(DAV, "resourcetype"),
types.iter().map(|(ns, l)| el(ns, l)),
)
};
let principals = format!("{PIM}/principals/");
let mut out = vec![
href_prop(DAV, "current-user-principal", &self.me.principal),
href_prop(DAV, "principal-collection-set", &principals),
];
match res {
Res::Root => out.push(resourcetype(&[(DAV, "collection")])),
Res::Principals => out.extend([
resourcetype(&[(DAV, "collection")]),
privileges(Access::Read),
live_props(self.me, self.space, res)
}
}
/// Every live property of a resource, with its value.
fn live_props(me: &Me, space: Option<&Space>, res: &Res) -> Vec<Element> {
let text = |ns: &str, local: &str, v: &str| with_text(el(ns, local), v);
let href_prop = |ns: &str, local: &str, h: &str| with_children(el(ns, local), hrefs([h]));
let resourcetype = |types: &[(&str, &str)]| {
with_children(
el(DAV, "resourcetype"),
types.iter().map(|(ns, l)| el(ns, l)),
)
};
let principals = format!("{PIM}/principals/");
let mut out = vec![
href_prop(DAV, "current-user-principal", &me.principal),
href_prop(DAV, "principal-collection-set", &principals),
];
match res {
Res::Root => out.push(resourcetype(&[(DAV, "collection")])),
Res::Principals => out.extend([
resourcetype(&[(DAV, "collection")]),
privileges(Access::Read),
principal_reports(),
]),
Res::Principal(p) => {
// The own principal in the spelling of the request.
let href = match p.me {
true => me.principal.clone(),
false => principal_href(&p.path),
};
let addresses = p.addresses();
out.extend([
resourcetype(&[(DAV, "collection"), (DAV, "principal")]),
text(DAV, "displayname", &p.display),
href_prop(DAV, "principal-URL", &href),
with_children(
el(CALDAV, "calendar-user-address-set"),
hrefs(addresses.iter().map(String::as_str))
.into_iter()
.map(|h| with_attr(h, "preferred", "1")),
),
with_children(
el(CALSERVER, "email-address-set"),
[with_text(
el(CALSERVER, "email-address"),
mailto(&p.path, p.kind),
)],
),
text(CALDAV, "calendar-user-type", p.kind.as_str()),
privileges(if p.me { Access::Own } else { Access::Read }),
principal_reports(),
]),
Res::Principal(p) => {
// The own principal in the spelling of the request.
let href = match p.me {
true => self.me.principal.clone(),
false => principal_href(&p.path),
]);
let home = |kind: PimKind| {
let name = match p.me {
true => space.map_or(p.path.clone(), |s| s.path.clone()),
false => p.path.clone(),
};
let addresses = p.addresses();
out.extend([
resourcetype(&[(DAV, "collection"), (DAV, "principal")]),
text(DAV, "displayname", &p.display),
href_prop(DAV, "principal-URL", &href),
with_children(
el(CALDAV, "calendar-user-address-set"),
hrefs(addresses.iter().map(String::as_str))
.into_iter()
.map(|h| with_attr(h, "preferred", "1")),
),
with_children(
el(CALSERVER, "email-address-set"),
[with_text(
el(CALSERVER, "email-address"),
mailto(&p.path, p.kind),
)],
),
text(CALDAV, "calendar-user-type", p.kind.as_str()),
privileges(if p.me { Access::Own } else { Access::Read }),
principal_reports(),
]);
let home = |kind: PimKind| {
let name = match p.me {
true => self.space.map_or(p.path.clone(), |s| s.path.clone()),
false => p.path.clone(),
};
format!("{PIM}/{}/{}/", kind_segment(kind), seg(&name))
};
// Also for other accounts: python-caldav drops a search hit
// without one. Their homes still answer 403.
format!("{PIM}/{}/{}/", kind_segment(kind), seg(&name))
};
// Also for other accounts: python-caldav drops a search hit
// without one. Their homes still answer 403.
out.push(href_prop(
CALDAV,
"calendar-home-set",
&home(PimKind::Calendar),
));
if p.me {
let cal = home(PimKind::Calendar);
out.push(href_prop(
CALDAV,
"calendar-home-set",
&home(PimKind::Calendar),
"schedule-inbox-URL",
&format!("{cal}{INBOX}/"),
));
out.push(href_prop(
CALDAV,
"schedule-outbox-URL",
&format!("{cal}{OUTBOX}/"),
));
let book = home(PimKind::AddressBook);
out.push(href_prop(CARDDAV, "addressbook-home-set", &book));
out.push(href_prop(
CARDDAV,
"directory-gateway",
&format!("{book}{DIRECTORY_SLUG}/"),
));
if p.me {
let cal = home(PimKind::Calendar);
out.push(href_prop(
CALDAV,
"schedule-inbox-URL",
&format!("{cal}{INBOX}/"),
));
out.push(href_prop(
CALDAV,
"schedule-outbox-URL",
&format!("{cal}{OUTBOX}/"),
));
let book = home(PimKind::AddressBook);
out.push(href_prop(CARDDAV, "addressbook-home-set", &book));
out.push(href_prop(
CARDDAV,
"directory-gateway",
&format!("{book}{DIRECTORY_SLUG}/"),
));
}
}
Res::Home(owner, access, _) => out.extend([
resourcetype(&[(DAV, "collection")]),
href_prop(DAV, "owner", owner),
privileges(*access),
]),
Res::Collection(kind, col) => {
let c = &col.c;
let (types, desc) = match kind {
PimKind::Calendar => ((CALDAV, "calendar"), (CALDAV, "calendar-description")),
PimKind::AddressBook => (
(CARDDAV, "addressbook"),
(CARDDAV, "addressbook-description"),
),
};
out.extend([
resourcetype(&[(DAV, "collection"), types]),
href_prop(DAV, "owner", &col.owner),
privileges(col.access),
supported_reports(*kind),
text(CALSERVER, "getctag", &format!("{}-{}", c.id, c.seq)),
text(DAV, "sync-token", &sync_token(c.id, c.seq)),
text(
kind_ns(*kind),
"max-resource-size",
&MAX_RESOURCE_SIZE.to_string(),
),
]);
if let Some(v) = &c.displayname {
out.push(text(DAV, "displayname", v));
}
if let Some(v) = &c.description {
out.push(text(desc.0, desc.1, v));
}
match kind {
PimKind::Calendar => {
out.push(with_children(
el(CALDAV, "supported-calendar-component-set"),
c.components
.split(',')
.map(|comp| with_attr(el(CALDAV, "comp"), "name", comp)),
));
out.push(with_children(
el(CALDAV, "supported-calendar-data"),
[with_attr(
with_attr(
el(CALDAV, "calendar-data"),
"content-type",
"text/calendar",
),
"version",
"2.0",
)],
));
if let Some(v) = &c.color {
out.push(text(APPLE, "calendar-color", v));
}
if let Some(v) = &c.sort_order {
out.push(text(APPLE, "calendar-order", v));
}
if let Some(v) = &c.timezone {
out.push(text(CALDAV, "calendar-timezone", v));
}
out.push(with_children(
el(CALDAV, "schedule-calendar-transp"),
[el(
CALDAV,
if c.transparent {
"transparent"
} else {
"opaque"
},
)],
));
}
// 3.0 only: a client told of 4.0 writes 4.0 groups, which
// Apple Contacts on the same account cannot read. A 4.0
// PUT is still stored, and served as 4.0 on request.
PimKind::AddressBook => out.push(with_children(
el(CARDDAV, "supported-address-data"),
}
Res::Home(owner, access, _) => out.extend([
resourcetype(&[(DAV, "collection")]),
href_prop(DAV, "owner", owner),
privileges(*access),
]),
Res::Collection(kind, col) => {
let c = &col.c;
let (types, desc) = match kind {
PimKind::Calendar => ((CALDAV, "calendar"), (CALDAV, "calendar-description")),
PimKind::AddressBook => (
(CARDDAV, "addressbook"),
(CARDDAV, "addressbook-description"),
),
};
out.extend([
resourcetype(&[(DAV, "collection"), types]),
href_prop(DAV, "owner", &col.owner),
privileges(col.access),
supported_reports(*kind),
text(CALSERVER, "getctag", &format!("{}-{}", c.id, c.seq)),
text(DAV, "sync-token", &sync_token(c.id, c.seq)),
text(
kind_ns(*kind),
"max-resource-size",
&MAX_RESOURCE_SIZE.to_string(),
),
]);
if let Some(v) = &c.displayname {
out.push(text(DAV, "displayname", v));
}
if let Some(v) = &c.description {
out.push(text(desc.0, desc.1, v));
}
match kind {
PimKind::Calendar => {
out.push(with_children(
el(CALDAV, "supported-calendar-component-set"),
c.components
.split(',')
.map(|comp| with_attr(el(CALDAV, "comp"), "name", comp)),
));
out.push(with_children(
el(CALDAV, "supported-calendar-data"),
[with_attr(
with_attr(
el(CARDDAV, "address-data-type"),
"content-type",
"text/vcard",
),
with_attr(el(CALDAV, "calendar-data"), "content-type", "text/calendar"),
"version",
"3.0",
"2.0",
)],
)),
));
if let Some(v) = &c.color {
out.push(text(APPLE, "calendar-color", v));
}
if let Some(v) = &c.sort_order {
out.push(text(APPLE, "calendar-order", v));
}
if let Some(v) = &c.timezone {
out.push(text(CALDAV, "calendar-timezone", v));
}
out.push(with_children(
el(CALDAV, "schedule-calendar-transp"),
[el(
CALDAV,
if c.transparent {
"transparent"
} else {
"opaque"
},
)],
));
}
// 3.0 only: a client told of 4.0 writes 4.0 groups, which
// Apple Contacts on the same account cannot read. A 4.0
// PUT is still stored, and served as 4.0 on request.
PimKind::AddressBook => out.push(with_children(
el(CARDDAV, "supported-address-data"),
[with_attr(
with_attr(
el(CARDDAV, "address-data-type"),
"content-type",
"text/vcard",
),
"version",
"3.0",
)],
)),
}
Res::Inbox(col, default) => {
let c = &col.c;
out.extend([
resourcetype(&[(DAV, "collection"), (CALDAV, "schedule-inbox")]),
href_prop(DAV, "owner", &col.owner),
privilege_set(INBOX_PRIVILEGES),
report_set(&[
(CALDAV, "calendar-multiget"),
(CALDAV, "calendar-query"),
(DAV, "sync-collection"),
]),
text(CALSERVER, "getctag", &format!("{}-{}", c.id, c.seq)),
text(DAV, "sync-token", &sync_token(c.id, c.seq)),
]);
if let Some(v) = &c.displayname {
out.push(text(DAV, "displayname", v));
}
if let Some(h) = default {
out.push(href_prop(CALDAV, "schedule-default-calendar-URL", h));
}
}
Res::Inbox(col, default) => {
let c = &col.c;
out.extend([
resourcetype(&[(DAV, "collection"), (CALDAV, "schedule-inbox")]),
href_prop(DAV, "owner", &col.owner),
privilege_set(INBOX_PRIVILEGES),
report_set(&[
(CALDAV, "calendar-multiget"),
(CALDAV, "calendar-query"),
(DAV, "sync-collection"),
]),
text(CALSERVER, "getctag", &format!("{}-{}", c.id, c.seq)),
text(DAV, "sync-token", &sync_token(c.id, c.seq)),
]);
if let Some(v) = &c.displayname {
out.push(text(DAV, "displayname", v));
}
Res::Outbox(owner) => out.extend([
resourcetype(&[(DAV, "collection"), (CALDAV, "schedule-outbox")]),
href_prop(DAV, "owner", owner),
privilege_set(OUTBOX_PRIVILEGES),
]),
Res::Object(kind, o) => {
if let Some(tag) = &o.schedule_tag {
out.push(text(CALDAV, "schedule-tag", tag));
}
out.extend([
resourcetype(&[]),
text(DAV, "getetag", &o.etag),
text(DAV, "getcontenttype", &content_type(*kind, &o.component)),
text(DAV, "getcontentlength", &o.size.to_string()),
]);
if let Ok(t) = chrono::DateTime::parse_from_rfc3339(&o.modified_at) {
let http_date = t.to_utc().format("%a, %d %b %Y %H:%M:%S GMT").to_string();
out.push(text(DAV, "getlastmodified", &http_date));
}
if let Some(h) = default {
out.push(href_prop(CALDAV, "schedule-default-calendar-URL", h));
}
}
Res::Outbox(owner) => out.extend([
resourcetype(&[(DAV, "collection"), (CALDAV, "schedule-outbox")]),
href_prop(DAV, "owner", owner),
privilege_set(OUTBOX_PRIVILEGES),
]),
Res::Object(kind, o) => {
if let Some(tag) = &o.schedule_tag {
out.push(text(CALDAV, "schedule-tag", tag));
}
out.extend([
resourcetype(&[]),
text(DAV, "getetag", &o.etag),
text(DAV, "getcontenttype", &content_type(*kind, &o.component)),
text(DAV, "getcontentlength", &o.size.to_string()),
]);
if let Ok(t) = chrono::DateTime::parse_from_rfc3339(&o.modified_at) {
let http_date = t.to_utc().format("%a, %d %b %Y %H:%M:%S GMT").to_string();
out.push(text(DAV, "getlastmodified", &http_date));
}
}
out
}
out
}
impl Cx<'_> {
@@ -2077,31 +2155,48 @@ impl Cx<'_> {
.as_deref()
.and_then(zone::from_vtimezone)
.unwrap_or(Zone::Utc);
let out = Out {
cx: self,
let mut out = Out {
me: self.me.clone(),
space: self.space().clone(),
kind: *kind,
col: &col,
col: col.clone(),
expanded: 0,
};
match report {
Report::CalendarMultiget { props, hrefs }
| Report::AddressbookMultiget { props, hrefs } => {
let mut responses = Vec::new();
let members = self.generated_members(&col).await?;
let mut found = Vec::with_capacity(hrefs.len());
for href in hrefs {
let found = match self.own_object(*kind, &href) {
Some((slug, name)) if slug == col.slug => self.member(&col, &name).await?,
let hit = match self.own_object(*kind, &href) {
Some((slug, name)) if slug == col.slug => match &members {
Some(m) => m.get(&name).cloned(),
None => self.state.db.pim_object(col.id, &name).await?,
},
_ => None,
};
responses.push(match found {
// The href as the client wrote it, so it can match it.
Some((o, data)) => match out.object(&o, &data, &props, &floating) {
Ok(r) => xml::Response { href, ..r },
Err(TooManyInstances) => return Ok(too_many()),
},
None => xml::Response::status(href, 404),
});
found.push((href, hit));
}
Ok(multistatus(&responses, None))
blocking(move || -> Reply {
let mut responses = Vec::new();
for (href, hit) in found {
if out.full() {
responses.push(out.over_limit());
break;
}
responses.push(match hit {
// The href as the client wrote it, so it can match it.
Some((o, data)) => match out.object(&o, &data, &props, &floating) {
Ok(r) => xml::Response { href, ..r },
Err(TooManyInstances) => return Ok(too_many()),
},
None => xml::Response::status(href, 404),
});
}
Ok(multistatus(&responses, None))
})
.await
}
Report::CalendarQuery {
props,
@@ -2109,46 +2204,60 @@ impl Cx<'_> {
timezone,
} => {
let floating = timezone.unwrap_or(floating);
let mut responses = Vec::new();
for (o, data) in self.members(&col).await? {
let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref()) else {
continue;
};
if filter::matches_calendar(&cal, &filter, &floating) {
let members = self.members(&col).await?;
blocking(move || -> Reply {
let mut responses = Vec::new();
for (o, data) in members {
let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref())
else {
continue;
};
if !filter::matches_calendar(&cal, &filter, &floating) {
continue;
}
if out.full() {
responses.push(out.over_limit());
break;
}
match out.object(&o, &data, &props, &floating) {
Ok(r) => responses.push(r),
Err(TooManyInstances) => return Ok(too_many()),
}
}
}
Ok(multistatus(&responses, None))
Ok(multistatus(&responses, None))
})
.await
}
Report::AddressbookQuery {
props,
filter,
limit,
} => {
let mut responses = Vec::new();
let mut truncated = false;
for (o, data) in self.members(&col).await? {
let Ok(card) = VCard::parse(String::from_utf8_lossy(&data).as_ref()) else {
continue;
};
if !filter::matches_card(&card, &filter) {
continue;
}
if limit.is_some_and(|n| responses.len() >= n) {
truncated = true;
break;
let members = self.members(&col).await?;
blocking(move || -> Reply {
let mut responses = Vec::new();
let mut truncated = false;
for (o, data) in members {
let Ok(card) = VCard::parse(String::from_utf8_lossy(&data).as_ref()) else {
continue;
};
if !filter::matches_card(&card, &filter) {
continue;
}
if limit.is_some_and(|n| responses.len() >= n) {
truncated = true;
break;
}
if let Ok(r) = out.object(&o, &data, &props, &floating) {
responses.push(r);
}
}
if let Ok(r) = out.object(&o, &data, &props, &floating) {
responses.push(r);
if truncated {
responses.push(out.over_limit());
}
}
if truncated {
responses.push(out.over_limit());
}
Ok(multistatus(&responses, None))
Ok(multistatus(&responses, None))
})
.await
}
Report::SyncCollection {
token,
@@ -2171,18 +2280,19 @@ impl Cx<'_> {
}
},
};
let mut changes = if generated(col.id) {
match since {
Some(_) => Vec::new(),
None => self
.members(&col)
.await?
.into_iter()
.map(|(o, _)| (o.name, col.seq, false))
.collect(),
// An initial sync reads every member at once, not one per change.
let mut members = match since {
None => Some(self.member_map(&col).await?),
Some(_) => None,
};
let mut changes = match (&members, generated(col.id)) {
(Some(m), true) => {
let mut names: Vec<_> = m.keys().cloned().collect();
names.sort();
names.into_iter().map(|n| (n, col.seq, false)).collect()
}
} else {
self.state.db.pim_changes(col.id, since).await?
(None, true) => Vec::new(),
(_, false) => self.state.db.pim_changes(col.id, since).await?,
};
let truncated = limit.is_some_and(|n| changes.len() > n);
if let Some(n) = limit {
@@ -2195,45 +2305,60 @@ impl Cx<'_> {
_ if generated(col.id) => col.seq,
(_, last) => col.seq.max(last.map_or(0, |(_, s, _)| *s)),
};
let mut responses = Vec::new();
let mut found = Vec::with_capacity(changes.len());
for (name, _, deleted) in changes {
let href = self.space().object(*kind, &col.slug, &name);
let found = match deleted {
true => None,
false => self.member(&col, &name).await?,
let hit = match (deleted, &mut members) {
(true, _) => None,
(false, Some(m)) => m.remove(&name),
(false, None) => self.state.db.pim_object(col.id, &name).await?,
};
responses.push(match found {
Some((o, data)) => match out.object(&o, &data, &props, &floating) {
Ok(r) => r,
Err(TooManyInstances) => return Ok(too_many()),
},
None => xml::Response::status(href, 404),
});
found.push((name, hit));
}
if truncated {
responses.push(out.over_limit());
}
Ok(multistatus(
&responses,
Some(with_text(el(DAV, "sync-token"), sync_token(col.id, seq))),
))
let slug = col.slug.clone();
blocking(move || -> Reply {
let mut responses = Vec::new();
for (name, hit) in found {
responses.push(match hit {
Some((o, data)) => match out.object(&o, &data, &props, &floating) {
Ok(r) => r,
Err(TooManyInstances) => return Ok(too_many()),
},
None => {
xml::Response::status(out.space.object(out.kind, &slug, &name), 404)
}
});
}
if truncated {
responses.push(out.over_limit());
}
Ok(multistatus(
&responses,
Some(with_text(el(DAV, "sync-token"), sync_token(col.id, seq))),
))
})
.await
}
Report::FreeBusy(range) => {
let mut busy = Vec::new();
for (_, data) in self.members(&col).await? {
if let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref()) {
// ponytail: one period per instance, so a long range over
// a frequent series makes a long answer.
busy.extend(freebusy::busy(&cal, &range, &floating, None));
let members = self.members(&col).await?;
blocking(move || -> Reply {
let mut busy = Vec::new();
for (_, data) in members {
if let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref()) {
// ponytail: one period per instance, so a long range over
// a frequent series makes a long answer.
busy.extend(freebusy::busy(&cal, &range, &floating, None));
}
}
}
let body = freebusy::vfreebusy(&freebusy::merge(busy), &range, chrono::Utc::now());
Ok((
StatusCode::OK,
[(CONTENT_TYPE, "text/calendar; charset=utf-8")],
body,
)
.into_response())
let body =
freebusy::vfreebusy(&freebusy::merge(busy), &range, chrono::Utc::now());
Ok((
StatusCode::OK,
[(CONTENT_TYPE, "text/calendar; charset=utf-8")],
body,
)
.into_response())
})
.await
}
Report::PrincipalSearch(_) | Report::PrincipalSearchPropertySet => {
unreachable!("answered above")
@@ -2315,25 +2440,38 @@ fn search_property_set() -> Response<Body> {
xml_response(StatusCode::OK, body)
}
/// What a REPORT answer about one collection needs.
struct Out<'a> {
cx: &'a Cx<'a>,
/// Instances `expand` may produce for one REPORT answer, across its objects.
/// Beyond it the answer is cut short with a 507, as for a client limit.
const MAX_EXPANDED_PER_ANSWER: usize = 20_000;
/// What a REPORT answer about one collection needs. Owned, so the answer
/// can be built on the blocking pool.
struct Out {
me: Me,
space: Space,
kind: PimKind,
col: &'a PimCollection,
col: PimCollection,
/// Instances `expand` produced for this answer so far.
expanded: usize,
}
impl Out<'_> {
impl Out {
fn object(
&self,
&mut self,
o: &PimObject,
data: &[u8],
props: &Props,
floating: &Zone,
) -> Result<xml::Response, TooManyInstances> {
let mut all = self.cx.props(&Res::Object(self.kind, o.clone()));
let mut all = live_props(
&self.me,
Some(&self.space),
&Res::Object(self.kind, o.clone()),
);
let raw = String::from_utf8_lossy(data);
if let Some(req) = &props.calendar {
let text = render::calendar_data(&raw, req, floating)?;
let (text, instances) = render::calendar_data(&raw, req, floating)?;
self.expanded += instances;
all.push(with_text(el(CALDAV, "calendar-data"), text));
}
if let Some(req) = &props.address {
@@ -2342,13 +2480,17 @@ impl Out<'_> {
render::address_data(&raw, req),
));
}
let href = self.cx.space().object(self.kind, &self.col.slug, &o.name);
let href = self.space.object(self.kind, &self.col.slug, &o.name);
Ok(select(href, &props.find, all))
}
/// The response a query or sync adds when a client limit cut it short.
fn full(&self) -> bool {
self.expanded > MAX_EXPANDED_PER_ANSWER
}
/// The response a query or sync adds when a limit cut it short.
fn over_limit(&self) -> xml::Response {
let href = self.cx.space().collection(self.kind, &self.col.slug);
let href = self.space.collection(self.kind, &self.col.slug);
let mut r = xml::Response::status(href, 507);
r.error = Some(el(DAV, "number-of-matches-within-limits"));
r
Mserver/src/api/pim_api.rs
@@ -34,7 +34,7 @@ use pimdav::bundle::{self, Detail};
use pimdav::{contact, object};
use sha2::{Digest, Sha256};
use crate::api::common::{SessionUser, hash_password, validate_password};
use crate::api::common::{SessionUser, blocking, hash_password, validate_password};
use crate::api::dav::challenge;
use crate::api::files::disposition;
use crate::api::pim::{
@@ -706,15 +706,19 @@ async fn render(
detail: Detail,
) -> Result<String, ApiError> {
let objects = members_of(state, owner, col.id).await?;
let texts: Vec<String> = objects
.into_iter()
.map(|(_, d)| String::from_utf8_lossy(&d).into_owned())
.collect();
let texts: Vec<&str> = texts.iter().map(String::as_str).collect();
Ok(match kind {
PimKind::Calendar => bundle::calendar(&texts, Some(&name_of(col)), detail),
PimKind::AddressBook => bundle::cards(&texts),
let name = name_of(col);
blocking(move || -> Result<String, ApiError> {
let texts: Vec<String> = objects
.into_iter()
.map(|(_, d)| String::from_utf8_lossy(&d).into_owned())
.collect();
let texts: Vec<&str> = texts.iter().map(String::as_str).collect();
Ok(match kind {
PimKind::Calendar => bundle::calendar(&texts, Some(&name), detail),
PimKind::AddressBook => bundle::cards(&texts),
})
})
.await
}
/// GET {PIM_COLLECTIONS}/{id}{EXPORT_SUFFIX}
@@ -738,7 +742,8 @@ pub async fn export_system(
}
async fn system_cards(state: &AppState) -> Result<(PimCollection, String), ApiError> {
let (col, members) = crate::api::pim::directory(state).await?;
let col = crate::api::pim::directory_collection(state).await?;
let members = crate::api::pim::directory(state).await?;
let texts: Vec<String> = members
.into_iter()
.map(|(_, d)| String::from_utf8_lossy(&d).into_owned())
Mserver/src/api/pim_schedule.rs
@@ -25,6 +25,7 @@ use super::pim::{
INBOX, MAIL_DOMAIN, OUTBOX, collection_href, etag_of, local_part, need_privilege,
principal_name, principal_uuid, seg,
};
use crate::api::common::blocking;
use crate::db::{PimKind, PimObject, PimOp, PimPrincipal};
use crate::error::{ApiError, AppState};
@@ -70,6 +71,7 @@ impl Writer<'_> {
}
/// Every principal, for mapping calendar user addresses.
#[derive(Clone)]
pub(crate) struct Directory(Vec<PimPrincipal>);
enum Recipient<'a> {
@@ -457,23 +459,31 @@ pub(crate) async fn busy_of(
range: &TimeRange,
skip: Option<&str>,
) -> Result<Vec<Period>, ApiError> {
let me = dir.is(p.id);
let mut busy = Vec::new();
let mut calendars = Vec::new();
for c in state.db.pim_collections(p.id, PimKind::Calendar).await? {
if c.slug == INBOX || c.transparent || !c.components.split(',').any(|x| x == "VEVENT") {
continue;
}
let floating = floating_of(c.timezone.as_deref());
for (o, data) in state.db.pim_objects_with_data(c.id).await? {
if skip.is_some_and(|u| u == o.uid) {
continue;
}
if let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref()) {
busy.extend(freebusy::busy(&cal, range, &floating, Some(&me)));
let objects = state.db.pim_objects_with_data(c.id).await?;
calendars.push((floating_of(c.timezone.as_deref()), objects));
}
let (dir, id, range, skip) = (dir.clone(), p.id, range.clone(), skip.map(str::to_string));
blocking(move || -> Result<_, ApiError> {
let me = dir.is(id);
let mut busy = Vec::new();
for (floating, objects) in calendars {
for (o, data) in objects {
if skip.as_deref().is_some_and(|u| u == o.uid) {
continue;
}
if let Ok(cal) = ICalendar::parse(String::from_utf8_lossy(&data).as_ref()) {
busy.extend(freebusy::busy(&cal, &range, &floating, Some(&me)));
}
}
}
}
Ok(freebusy::merge(busy))
Ok(freebusy::merge(busy))
})
.await
}
fn floating_of(timezone: Option<&str>) -> Zone {
Mserver/src/api/pim_views.rs
@@ -28,7 +28,8 @@ use pimdav::zone::{self, Zone};
use serde::Deserialize;
use crate::api::common::SessionUser;
use crate::api::pim::{BIRTHDAYS, DIRECTORY, INBOX, etag_of, mailto, members_of, seg};
use crate::api::common::blocking;
use crate::api::pim::{BIRTHDAYS, DIRECTORY, INBOX, etag_of, generated, mailto, members_of, seg};
use crate::api::pim_api::reachable;
use crate::api::pim_schedule::{self, Directory, Writer};
use crate::db::{PimKind, PimObject, PimOp};
@@ -136,48 +137,56 @@ pub async fn instances(
let zone = floating(q.tz.as_deref());
let wanted = wanted(q.collections.as_deref());
let dir = Directory::load(&state).await?;
let mut out = Vec::new();
let mut truncated = false;
'all: for (id, owner) in readable(&state, &auth, PimKind::Calendar).await? {
let mut sources = Vec::new();
for (id, owner) in readable(&state, &auth, PimKind::Calendar).await? {
if wanted.as_ref().is_some_and(|w| !w.contains(&id)) {
continue;
}
let owns = dir.is(owner);
for (obj, data) in members_of(&state, owner, id).await? {
let Some(cal) = parse(&data) else {
continue;
};
let exp = expand(&cal, from..to, zone.clone());
truncated |= exp.truncated;
let mut infos: HashMap<usize, EventInfo> = HashMap::new();
for i in exp.instances {
if out.len() == MAX_INSTANCES {
truncated = true;
break 'all;
sources.push((id, owner, members_of(&state, owner, id).await?));
}
let (mut out, truncated) = blocking(move || -> Result<_, ApiError> {
let mut out = Vec::new();
let mut truncated = false;
'all: for (id, owner, members) in sources {
let owns = dir.is(owner);
for (obj, data) in members {
let Some(cal) = parse(&data) else {
continue;
};
let exp = expand(&cal, from..to, zone.clone());
truncated |= exp.truncated;
let mut infos: HashMap<usize, EventInfo> = HashMap::new();
for i in exp.instances {
if out.len() == MAX_INSTANCES {
truncated = true;
break 'all;
}
let info = infos
.entry(i.component)
.or_insert_with(|| view::event_info(&cal, i.component, &owns));
out.push(PimInstance {
collection_id: id,
name: obj.name.clone(),
uid: obj.uid.clone(),
recurrence_id: i.recurrence_id.map(rfc3339),
start: rfc3339(i.start),
end: rfc3339(i.end),
all_day: info.all_day,
component: info.component.clone(),
summary: info.summary.clone(),
location: info.location.clone(),
status: info.status.clone(),
transparent: info.transparent,
has_attendees: !info.attendees.is_empty(),
partstat: info.partstat().map(str::to_string),
organizer: info.organizer.as_ref().map(display),
});
}
let info = infos
.entry(i.component)
.or_insert_with(|| view::event_info(&cal, i.component, &owns));
out.push(PimInstance {
collection_id: id,
name: obj.name.clone(),
uid: obj.uid.clone(),
recurrence_id: i.recurrence_id.map(rfc3339),
start: rfc3339(i.start),
end: rfc3339(i.end),
all_day: info.all_day,
component: info.component.clone(),
summary: info.summary.clone(),
location: info.location.clone(),
status: info.status.clone(),
transparent: info.transparent,
has_attendees: !info.attendees.is_empty(),
partstat: info.partstat().map(str::to_string),
organizer: info.organizer.as_ref().map(display),
});
}
}
}
Ok((out, truncated))
})
.await?;
out.sort_by(|a, b| (&a.start, &a.end).cmp(&(&b.start, &b.end)));
Ok(Json(PimInstances {
instances: out,
@@ -200,11 +209,14 @@ pub async fn object(
) -> Result<Json<PimObjectDetail>, ApiError> {
let not_found = || ApiError::new(StatusCode::NOT_FOUND, "object not found");
let (owner, kind, col, writable) = reachable(&state, &auth, id).await?;
let members = members_of(&state, owner, col.id).await?;
let (obj, data) = members
.iter()
.find(|(o, _)| o.name == name)
.ok_or_else(not_found)?;
let found = match generated(col.id) {
true => members_of(&state, owner, col.id)
.await?
.into_iter()
.find(|(o, _)| o.name == name),
false => state.db.pim_object(col.id, &name).await?,
};
let (obj, data) = &found.ok_or_else(not_found)?;
match kind {
PimKind::Calendar => {
let cal = parse(data).ok_or_else(not_found)?;
@@ -249,7 +261,8 @@ pub async fn object(
let card = view::card(&String::from_utf8_lossy(data));
let members = match card.is_group {
true => {
let by_uid: HashMap<String, String> = members
let by_uid: HashMap<String, String> = members_of(&state, owner, col.id)
.await?
.iter()
.map(|(_, d)| view::card(&String::from_utf8_lossy(d)))
.filter_map(|c| Some((c.uid?, c.full_name)))
@@ -347,35 +360,43 @@ pub async fn contacts(
) -> Result<Json<Vec<PimContact>>, ApiError> {
let needle = q.q.as_deref().map(str::trim).unwrap_or("").to_lowercase();
let wanted = wanted(q.collections.as_deref());
let mut out = Vec::new();
let mut sources = Vec::new();
for (id, owner) in readable(&state, &auth, PimKind::AddressBook).await? {
if wanted.as_ref().is_some_and(|w| !w.contains(&id)) {
continue;
}
for (obj, data) in members_of(&state, owner, id).await? {
let c = view::card(&String::from_utf8_lossy(&data));
let hit = needle.is_empty()
|| [Some(&c.full_name), c.org.as_ref()]
.into_iter()
.flatten()
.chain(c.emails.iter().map(|e| &e.value))
.chain(c.phones.iter().map(|p| &p.value))
.any(|v| v.to_lowercase().contains(&needle));
if !hit {
continue;
sources.push((id, members_of(&state, owner, id).await?));
}
let mut out = blocking(move || -> Result<_, ApiError> {
let mut out = Vec::new();
for (id, members) in sources {
for (obj, data) in members {
let c = view::card(&String::from_utf8_lossy(&data));
let hit = needle.is_empty()
|| [Some(&c.full_name), c.org.as_ref()]
.into_iter()
.flatten()
.chain(c.emails.iter().map(|e| &e.value))
.chain(c.phones.iter().map(|p| &p.value))
.any(|v| v.to_lowercase().contains(&needle));
if !hit {
continue;
}
out.push(PimContact {
collection_id: id,
name: obj.name,
full_name: c.full_name,
org: c.org,
email: c.emails.into_iter().next().map(|e| e.value),
phone: c.phones.into_iter().next().map(|p| p.value),
has_photo: c.has_photo,
is_group: c.is_group,
});
}
out.push(PimContact {
collection_id: id,
name: obj.name,
full_name: c.full_name,
org: c.org,
email: c.emails.into_iter().next().map(|e| e.value),
phone: c.phones.into_iter().next().map(|p| p.value),
has_photo: c.has_photo,
is_group: c.is_group,
});
}
}
Ok(out)
})
.await?;
out.sort_by_cached_key(|c| (c.full_name.to_lowercase(), c.collection_id));
Ok(Json(out))
}
Mserver/src/db.rs
@@ -367,7 +367,12 @@ const USER_COL_COUNT: usize = 11;
pub type DbResult<T> = Result<T, rusqlite::Error>;
#[derive(Clone)]
pub struct Db(Arc<tokio::sync::Mutex<Connection>>);
pub struct Db {
conn: Arc<tokio::sync::Mutex<Connection>>,
/// Principals whose default collections exist. Saves the three writes
/// of [`Db::pim_ensure_defaults`] on every DAV request.
defaults: Arc<std::sync::Mutex<std::collections::HashSet<i64>>>,
}
impl Db {
pub async fn open(path: &Path) -> anyhow::Result<Self> {
@@ -384,7 +389,10 @@ impl Db {
conn.pragma_update(None, "foreign_keys", "ON")?;
conn.pragma_update(None, "busy_timeout", "5000")?;
Self::migrate(&conn)?;
Ok(Self(Arc::new(tokio::sync::Mutex::new(conn))))
Ok(Self {
conn: Arc::new(tokio::sync::Mutex::new(conn)),
defaults: Default::default(),
})
}
fn migrate(conn: &Connection) -> rusqlite::Result<()> {
@@ -625,7 +633,7 @@ impl Db {
// ---------- users ----------
pub async fn user_count(&self) -> DbResult<i64> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached("SELECT COUNT(*) FROM users")?;
stmt.query_row([], |r| r.get(0))
}
@@ -637,7 +645,7 @@ impl Db {
/// setups cannot both win; a caller's earlier `user_count` check is only
/// an optimization, not the guarantee.
pub async fn create_admin(&self, name: &str, pass_hash: &str) -> DbResult<Option<User>> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let inserted = tx.execute(
"INSERT INTO users (name, pass_hash, is_admin, created_at)
@@ -661,12 +669,11 @@ impl Db {
// design; holding the single connection lock across it would make one
// login serialize every other database access.
let row: Option<(User, String)> = {
let c = self.0.lock().await;
c.query_row(
&format!("SELECT {USER_COLS}, pass_hash FROM users WHERE name = ?1"),
[name],
|r| Ok((map_user(r)?, r.get(USER_COL_COUNT)?)),
)
let c = self.conn.lock().await;
c.prepare_cached(&format!(
"SELECT {USER_COLS}, pass_hash FROM users WHERE name = ?1"
))?
.query_row([name], |r| Ok((map_user(r)?, r.get(USER_COL_COUNT)?)))
.optional()?
};
// An unknown name, a disabled account and a passkey-only account all
@@ -681,7 +688,7 @@ impl Db {
}
pub async fn create_session(&self, user_id: i64, token: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"INSERT INTO sessions (token, user_id, created_at, last_seen_at)
VALUES (?1, ?2, ?3, ?4)",
@@ -691,7 +698,7 @@ impl Db {
}
pub async fn delete_session(&self, token: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute("DELETE FROM sessions WHERE token = ?1", [token])?;
Ok(())
}
@@ -702,7 +709,7 @@ impl Db {
&self,
token: &str,
) -> DbResult<Option<(User, Vec<RootRow>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {USER_COLS_U}, r.id, r.path, r.mode
FROM sessions s
@@ -734,7 +741,7 @@ impl Db {
// ---------- roots ----------
pub async fn user_roots(&self, user_id: i64) -> DbResult<Vec<RootRow>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT id, path, mode FROM user_roots WHERE user_id = ?1 ORDER BY id",
)?;
@@ -753,7 +760,7 @@ impl Db {
/// Every user with their roots, in one query. The admin user list needs
/// both, and a per-user roots query would be one round trip per user.
pub async fn all_users_with_roots(&self) -> DbResult<Vec<(User, Vec<RootRow>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {USER_COLS_U}, r.id, r.path, r.mode
FROM users u
@@ -780,17 +787,14 @@ impl Db {
}
pub async fn find_user_by_id(&self, id: i64) -> DbResult<Option<User>> {
let c = self.0.lock().await;
c.query_row(
&format!("SELECT {USER_COLS} FROM users WHERE id = ?1"),
[id],
map_user,
)
.optional()
let c = self.conn.lock().await;
c.prepare_cached(&format!("SELECT {USER_COLS} FROM users WHERE id = ?1"))?
.query_row([id], map_user)
.optional()
}
pub async fn find_user_by_name(&self, name: &str) -> DbResult<Option<User>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.query_row(
&format!("SELECT {USER_COLS} FROM users WHERE name = ?1"),
[name],
@@ -800,7 +804,7 @@ impl Db {
}
pub async fn count_admins(&self) -> DbResult<i64> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.query_row(
"SELECT COUNT(*) FROM users WHERE is_admin = 1 AND active = 1",
[],
@@ -816,7 +820,7 @@ impl Db {
is_admin: bool,
roots: &[(String, Mode)],
) -> DbResult<User> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
tx.execute(
"INSERT INTO users (name, pass_hash, is_admin, active, created_at)
@@ -836,7 +840,7 @@ impl Db {
/// Write the profile settings a user edits for themselves.
pub async fn set_user_profile(&self, u: &User) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"UPDATE users SET single_click = ?1, thumbnails = ?2, language = ?3,
default_root_id = ?4, week_start = ?5
@@ -863,7 +867,7 @@ impl Db {
active: Option<bool>,
roots: Option<&[(String, Mode)]>,
) -> DbResult<()> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
// An admin sets a password to get someone back into a locked-out
// account, so every other way in goes with it: the passkeys, the app
@@ -913,7 +917,7 @@ impl Db {
/// Only for setting a real one. Clearing it is [`Db::clear_user_password`],
/// which has a rule to keep.
pub async fn set_password_keeping_sessions(&self, id: i64, pass_hash: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"UPDATE users SET pass_hash = ?1 WHERE id = ?2",
params![pass_hash, id],
@@ -924,7 +928,7 @@ impl Db {
/// Leave the account on its passkeys alone. `false` means that would have
/// locked it out, so nothing changed.
pub async fn clear_user_password(&self, id: i64) -> DbResult<bool> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
tx.execute(
"UPDATE users SET pass_hash = ?1 WHERE id = ?2",
@@ -935,7 +939,7 @@ impl Db {
/// `false` means the account does not satisfy the new mode, so it stands.
pub async fn set_user_auth_mode(&self, id: i64, mode: AuthMode) -> DbResult<bool> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
tx.execute(
"UPDATE users SET auth_mode = ?1 WHERE id = ?2",
@@ -949,7 +953,7 @@ impl Db {
/// Called after any credential change. Otherwise a session stolen before
/// the change keeps working for its full 30 days.
pub async fn delete_other_sessions(&self, user_id: i64, keep: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"DELETE FROM sessions WHERE user_id = ?1 AND token != ?2",
params![user_id, keep],
@@ -963,7 +967,7 @@ impl Db {
/// this value and hands it back at sign-in, so changing it would orphan
/// every existing passkey.
pub async fn user_webauthn_id(&self, id: i64) -> DbResult<Uuid> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let existing: Option<String> = c
.query_row("SELECT webauthn_id FROM users WHERE id = ?1", [id], |r| {
r.get(0)
@@ -983,7 +987,7 @@ impl Db {
/// The account a discoverable credential's user handle points at.
pub async fn find_user_by_webauthn_id(&self, wid: &Uuid) -> DbResult<Option<User>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.query_row(
&format!("SELECT {USER_COLS} FROM users WHERE webauthn_id = ?1"),
[wid.to_string()],
@@ -993,7 +997,7 @@ impl Db {
}
pub async fn user_passkeys(&self, user_id: i64) -> DbResult<Vec<PasskeyRow>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT id, name, created_at, last_used_at, discoverable, passkey
FROM passkeys WHERE user_id = ?1 ORDER BY id",
@@ -1003,7 +1007,7 @@ impl Db {
}
pub async fn count_passkeys(&self, user_id: i64) -> DbResult<i64> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.query_row(
"SELECT COUNT(*) FROM passkeys WHERE user_id = ?1",
[user_id],
@@ -1014,7 +1018,7 @@ impl Db {
/// The per-install secret behind the decoy credentials a named passkey
/// challenge is padded with. Created on first use, so no migration.
pub async fn decoy_secret(&self) -> DbResult<String> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let existing: Option<String> = c
.query_row(
"SELECT value FROM meta WHERE key = 'decoy_secret'",
@@ -1044,7 +1048,7 @@ impl Db {
/// would make a rare length as likely as a common one, and decoys that do
/// not match how the install actually looks are the thing worth avoiding.
pub async fn cred_id_lengths(&self) -> DbResult<Vec<usize>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare("SELECT length(cred_id) FROM passkeys ORDER BY id")?;
let rows = stmt.query_map([], |r| r.get::<_, i64>(0))?;
rows.map(|r| r.map(|n| n.max(1) as usize))
@@ -1067,7 +1071,7 @@ impl Db {
name: &str,
discoverable: Option<bool>,
) -> DbResult<Option<PasskeyRow>> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let held: i64 = tx.query_row(
"SELECT COUNT(*) FROM passkeys WHERE user_id = ?1",
@@ -1094,7 +1098,7 @@ impl Db {
}
pub async fn delete_passkey(&self, id: i64, user_id: i64) -> DbResult<PasskeyDeleted> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let hit = tx.execute(
"DELETE FROM passkeys WHERE id = ?1 AND user_id = ?2",
@@ -1114,7 +1118,7 @@ impl Db {
/// Record a successful assertion: the re-serialized credential (its
/// signature counter and backup flags may have moved) and the time.
pub async fn passkey_used(&self, id: i64, passkey: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"UPDATE passkeys SET passkey = ?1, last_used_at = ?2 WHERE id = ?3",
params![passkey, now(), id],
@@ -1125,7 +1129,7 @@ impl Db {
// ---------- app passwords (WebDAV) ----------
pub async fn app_passwords(&self, user_id: i64) -> DbResult<Vec<AppPasswordInfo>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT id, name, created_at, last_used_at
FROM app_passwords WHERE user_id = ?1 ORDER BY id",
@@ -1142,7 +1146,7 @@ impl Db {
name: &str,
secret_hash: &str,
) -> DbResult<Option<AppPasswordInfo>> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let held: i64 = tx.query_row(
"SELECT COUNT(*) FROM app_passwords WHERE user_id = ?1",
@@ -1171,7 +1175,7 @@ impl Db {
/// No reachability check, unlike [`Db::delete_passkey`]: an app password
/// never signs in to the web UI.
pub async fn delete_app_password(&self, id: i64, user_id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute(
"DELETE FROM app_passwords WHERE id = ?1 AND user_id = ?2",
params![id, user_id],
@@ -1183,18 +1187,15 @@ impl Db {
/// Inactive accounts are excluded here, because nothing downstream in the
/// WebDAV path looks at the flag.
pub async fn user_by_app_password(&self, secret_hash: &str) -> DbResult<Option<User>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let user = c
.query_row(
&format!(
"SELECT {USER_COLS_U}
FROM app_passwords a
JOIN users u ON u.id = a.user_id
WHERE a.secret_hash = ?1 AND u.active = 1"
),
[secret_hash],
map_user,
)
.prepare_cached(&format!(
"SELECT {USER_COLS_U}
FROM app_passwords a
JOIN users u ON u.id = a.user_id
WHERE a.secret_hash = ?1 AND u.active = 1"
))?
.query_row([secret_hash], map_user)
.optional()?;
if user.is_some() {
// A mount re-sends its credential on every request. An hour's
@@ -1212,7 +1213,7 @@ impl Db {
/// The stored spelling of the name, which a Basic login may differ from
/// in case.
pub async fn user_name(&self, id: i64) -> DbResult<Option<String>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached("SELECT name FROM users WHERE id = ?1")?;
stmt.query_row([id], |r| r.get(0)).optional()
}
@@ -1220,11 +1221,12 @@ impl Db {
/// Deletes an account after `ops`, in one transaction. The ops make other
/// principals' objects forget it (`pim_schedule::forget`).
pub async fn delete_user(&self, id: i64, ops: &[PimOp]) -> DbResult<bool> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
apply_ops(&tx, ops)?;
let deleted = tx.execute("DELETE FROM users WHERE id = ?1", [id])? > 0;
tx.commit()?;
self.forget_defaults();
Ok(deleted)
}
@@ -1241,7 +1243,7 @@ impl Db {
expires_at: Option<&str>,
password_hash: Option<&str>,
) -> DbResult<ShareRow> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"INSERT INTO shares
(token, creator_id, target, is_file, mode, created_at, expires_at, password_hash)
@@ -1275,7 +1277,7 @@ impl Db {
/// token that proves it (the value of their unlock cookie).
pub async fn create_share_unlock(&self, share_id: i64) -> DbResult<String> {
let token = crate::auth::random_token();
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"INSERT INTO share_unlocks (token, share_id, created_at) VALUES (?1, ?2, ?3)",
params![token, share_id, now()],
@@ -1289,7 +1291,7 @@ impl Db {
/// Housekeeping only: every read already refuses an expired share, so
/// nothing here is load-bearing and the interval does not matter.
pub async fn sweep(&self) -> DbResult<(usize, usize)> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
// SQLite parses the timestamp instead of comparing it as text:
// `expires_at` is stored exactly as the client sent it and may carry
// an offset or fractional seconds. An unparseable one yields NULL and
@@ -1318,7 +1320,7 @@ impl Db {
/// The share id is part of the lookup, so an unlock for one share cannot
/// open another.
pub async fn share_unlock_valid(&self, token: &str, share_id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt =
c.prepare_cached("SELECT 1 FROM share_unlocks WHERE token = ?1 AND share_id = ?2")?;
Ok(stmt
@@ -1328,7 +1330,7 @@ impl Db {
}
pub async fn share_by_token(&self, token: &str) -> DbResult<Option<ShareRow>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let sql = "SELECT id, token, creator_id, target, is_file, mode, created_at, expires_at,
password_hash
FROM shares WHERE token = ?1";
@@ -1337,7 +1339,7 @@ impl Db {
}
pub async fn user_shares(&self, creator_id: i64) -> DbResult<Vec<ShareRow>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let sql = "SELECT id, token, creator_id, target, is_file, mode, created_at, expires_at,
password_hash
FROM shares WHERE creator_id = ?1 ORDER BY id DESC";
@@ -1357,7 +1359,7 @@ impl Db {
/// `substr` rather than `LIKE`: a target containing `%` or `_` would make
/// a `LIKE` pattern over-match and revoke unrelated shares.
pub async fn revoke_shares_at(&self, target: &str) -> DbResult<usize> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"DELETE FROM shares
WHERE target = ?1 OR substr(target, 1, length(?1) + 1) = ?1 || '/'",
@@ -1367,7 +1369,7 @@ impl Db {
/// Delete one of `creator_id`'s shares. `false` means no row matched.
pub async fn delete_share(&self, id: i64, creator_id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let n = c.execute(
"DELETE FROM shares WHERE id = ?1 AND creator_id = ?2",
params![id, creator_id],
@@ -1381,7 +1383,7 @@ impl Db {
/// The join cannot miss: `shares.creator_id` cascades on delete, so a share
/// never outlives the account that made it.
pub async fn all_shares_with_creators(&self) -> DbResult<Vec<ShareWithCreator>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
// Columns 0..8 are `map_share`'s order, unchanged from `user_shares`.
let sql = "SELECT s.id, s.token, s.creator_id, s.target, s.is_file, s.mode,
s.created_at, s.expires_at, s.password_hash,
@@ -1403,7 +1405,7 @@ impl Db {
/// Revoke a share whoever created it. The owner-scoped
/// [`Self::delete_share`] is what the user-facing API uses.
pub async fn admin_delete_share(&self, id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute("DELETE FROM shares WHERE id = ?1", [id])? > 0)
}
@@ -1439,13 +1441,13 @@ impl Db {
}
pub async fn get_setting(&self, key: &str) -> DbResult<Option<String>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached("SELECT value FROM settings WHERE key = ?1")?;
stmt.query_row([key], |r| r.get(0)).optional()
}
pub async fn set_setting(&self, key: &str, value: &str) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"INSERT INTO settings (key, value) VALUES (?1, ?2)
ON CONFLICT(key) DO UPDATE SET value = ?2",
@@ -1459,7 +1461,10 @@ impl Db {
/// Gives an account's principal a calendar, an address book and a
/// scheduling inbox when it has none.
pub async fn pim_ensure_defaults(&self, principal_id: i64) -> DbResult<()> {
let c = self.0.lock().await;
if self.defaults_known(principal_id) {
return Ok(());
}
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"INSERT INTO pim_collections (principal_id, kind, slug, displayname, components, created_at)
SELECT ?1, ?2, 'default', ?3, ?4, ?5
@@ -1475,12 +1480,28 @@ impl Db {
now
])?;
stmt.execute(params![principal_id, "card", "Contacts", "", now])?;
ensure_inbox(&c, principal_id)
ensure_inbox(&c, principal_id)?;
self.defaults_lock().insert(principal_id);
Ok(())
}
fn defaults_lock(&self) -> std::sync::MutexGuard<'_, std::collections::HashSet<i64>> {
self.defaults.lock().unwrap_or_else(|e| e.into_inner())
}
fn defaults_known(&self, principal_id: i64) -> bool {
self.defaults_lock().contains(&principal_id)
}
/// After a collection or principal is deleted, defaults may be missing
/// again, and a principal id may be reused.
fn forget_defaults(&self) {
self.defaults_lock().clear();
}
/// The scheduling inbox alone, for rooms and resources.
pub async fn pim_ensure_inbox(&self, principal_id: i64) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
ensure_inbox(&c, principal_id)
}
@@ -1491,7 +1512,7 @@ impl Db {
principal_id: i64,
component: &str,
) -> DbResult<Option<PimCollection>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS} FROM pim_collections
WHERE principal_id = ?1 AND kind = 'cal' AND slug != 'inbox'
@@ -1508,7 +1529,7 @@ impl Db {
principal_id: i64,
uid: &str,
) -> DbResult<Option<(i64, PimObject, Vec<u8>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT o.name, o.uid, o.component, o.etag, length(o.data), o.modified_at,
o.schedule_tag, o.data, o.collection_id
@@ -1529,18 +1550,17 @@ impl Db {
uid: &str,
name: &str,
) -> DbResult<Option<String>> {
let c = self.0.lock().await;
c.query_row(
let c = self.conn.lock().await;
c.prepare_cached(
"SELECT name FROM pim_objects WHERE collection_id = ?1 AND uid = ?2 AND name != ?3",
params![collection_id, uid, name],
|r| r.get(0),
)
)?
.query_row(params![collection_id, uid, name], |r| r.get(0))
.optional()
}
/// Several object writes in one transaction.
pub async fn pim_apply(&self, ops: &[PimOp]) -> DbResult<()> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
apply_ops(&tx, ops)?;
tx.commit()?;
@@ -1552,7 +1572,7 @@ impl Db {
principal_id: i64,
kind: PimKind,
) -> DbResult<Vec<PimCollection>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS} FROM pim_collections
WHERE principal_id = ?1 AND kind = ?2 ORDER BY id"
@@ -1567,7 +1587,7 @@ impl Db {
kind: PimKind,
slug: &str,
) -> DbResult<Option<PimCollection>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS} FROM pim_collections
WHERE principal_id = ?1 AND kind = ?2 AND slug = ?3"
@@ -1587,7 +1607,7 @@ impl Db {
new: &PimCollection,
props: &[DeadProp],
) -> DbResult<bool> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let n = tx.execute(
"INSERT OR IGNORE INTO pim_collections (principal_id, kind, slug, displayname,
@@ -1625,7 +1645,7 @@ impl Db {
set: &[DeadProp],
remove: &[(String, String)],
) -> DbResult<()> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
if let Some(col) = col {
tx.execute(
@@ -1649,7 +1669,7 @@ impl Db {
}
pub async fn pim_props(&self, place: PropPlace) -> DbResult<Vec<DeadProp>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT ns, name, xml FROM pim_props WHERE place = ?1 ORDER BY rowid",
)?;
@@ -1664,13 +1684,14 @@ impl Db {
}
pub async fn pim_delete_collection(&self, id: i64) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute("DELETE FROM pim_collections WHERE id = ?1", [id])?;
self.forget_defaults();
Ok(())
}
pub async fn pim_objects(&self, collection_id: i64) -> DbResult<Vec<PimObject>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_OBJECT_COLS} FROM pim_objects WHERE collection_id = ?1 ORDER BY name"
))?;
@@ -1682,7 +1703,7 @@ impl Db {
collection_id: i64,
name: &str,
) -> DbResult<Option<(PimObject, Vec<u8>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_OBJECT_COLS}, data FROM pim_objects
WHERE collection_id = ?1 AND name = ?2"
@@ -1697,7 +1718,7 @@ impl Db {
&self,
collection_id: i64,
) -> DbResult<Vec<(PimObject, Vec<u8>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_OBJECT_COLS}, data FROM pim_objects WHERE collection_id = ?1 ORDER BY name"
))?;
@@ -1712,16 +1733,26 @@ impl Db {
collection_id: i64,
since: Option<i64>,
) -> DbResult<Vec<(String, i64, bool)>> {
let c = self.0.lock().await;
let mut stmt = c.prepare_cached(
"SELECT name, seq, deleted != 0 FROM pim_changes
WHERE collection_id = ?1 AND (?2 IS NULL AND deleted = 0 OR seq > ?2)
ORDER BY seq",
)?;
stmt.query_map(params![collection_id, since], |r| {
Ok((r.get(0)?, r.get(1)?, r.get(2)?))
})?
.collect()
let c = self.conn.lock().await;
// Two statements, not one with `?2 IS NULL OR ...`: the OR keeps
// SQLite from using the seq range of the index.
let map = |r: &rusqlite::Row| Ok((r.get(0)?, r.get(1)?, r.get(2)?));
match since {
None => c
.prepare_cached(
"SELECT name, seq, 0 FROM pim_changes
WHERE collection_id = ?1 AND deleted = 0 ORDER BY seq",
)?
.query_map([collection_id], map)?
.collect(),
Some(seq) => c
.prepare_cached(
"SELECT name, seq, deleted != 0 FROM pim_changes
WHERE collection_id = ?1 AND seq > ?2 ORDER BY seq",
)?
.query_map(params![collection_id, seq], map)?
.collect(),
}
}
/// Moves an object to `to_name` in collection `to`, which may be the
@@ -1736,7 +1767,7 @@ impl Db {
overwrite: bool,
cond: &Precondition,
) -> DbResult<PimWrite> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let source: Option<(String, String)> = tx
.query_row(
@@ -1794,7 +1825,7 @@ impl Db {
/// An account, room or resource by URL name. Disabled accounts are
/// invisible.
pub async fn pim_principal(&self, name: &str) -> DbResult<Option<PimPrincipal>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PRINCIPAL_COLS} FROM {PRINCIPALS} WHERE p.name = ?1 AND {VISIBLE}"
))?;
@@ -1802,7 +1833,7 @@ impl Db {
}
pub async fn pim_principals(&self) -> DbResult<Vec<PimPrincipal>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PRINCIPAL_COLS} FROM {PRINCIPALS} WHERE {VISIBLE} ORDER BY p.id"
))?;
@@ -1811,14 +1842,14 @@ impl Db {
/// The principal of an account.
pub async fn principal_of(&self, user_id: i64) -> DbResult<i64> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.prepare_cached("SELECT id FROM principals WHERE user_id = ?1")?
.query_row([user_id], |r| r.get(0))
}
/// Whether an account, room or resource has this name.
pub async fn name_taken(&self, name: &str) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.prepare_cached("SELECT EXISTS (SELECT 1 FROM principals WHERE name = ?1)")?
.query_row([name], |r| r.get(0))
}
@@ -1831,7 +1862,7 @@ impl Db {
kind: PimKind,
collection_id: i64,
) -> DbResult<Option<(PimCollection, String, PimShareMode)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS_C}, p.name, s.mode
FROM pim_shares s
@@ -1849,7 +1880,7 @@ impl Db {
user_id: i64,
kind: PimKind,
) -> DbResult<Vec<(PimCollection, String, PimShareMode)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS_C}, p.name, s.mode
FROM pim_shares s
@@ -1866,7 +1897,7 @@ impl Db {
&self,
id: i64,
) -> DbResult<Option<(i64, PimKind, PimCollection)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_COLLECTION_COLS}, principal_id, kind FROM pim_collections WHERE id = ?1"
))?;
@@ -1888,7 +1919,7 @@ impl Db {
expires_at: Option<&str>,
password_hash: Option<&str>,
) -> DbResult<PimLink> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let created_at = now();
c.execute(
"INSERT INTO pim_links
@@ -1915,7 +1946,7 @@ impl Db {
}
pub async fn pim_links(&self, collection_id: i64) -> DbResult<Vec<PimLink>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_LINK_COLS} FROM pim_links WHERE collection_id = ?1 ORDER BY id"
))?;
@@ -1923,7 +1954,7 @@ impl Db {
}
pub async fn pim_link_by_token(&self, token: &str) -> DbResult<Option<PimLink>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PIM_LINK_COLS} FROM pim_links WHERE token = ?1"
))?;
@@ -1932,7 +1963,7 @@ impl Db {
/// `false` means no link of that collection had the id.
pub async fn pim_delete_link(&self, collection_id: i64, id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute(
"DELETE FROM pim_links WHERE id = ?1 AND collection_id = ?2",
params![id, collection_id],
@@ -1944,7 +1975,7 @@ impl Db {
&self,
collection_id: i64,
) -> DbResult<Vec<(i64, String, PimShareMode)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT u.id, u.name, s.mode FROM pim_shares s JOIN users u ON u.id = s.user_id
WHERE s.collection_id = ?1 ORDER BY u.name",
@@ -1962,7 +1993,7 @@ impl Db {
collection_id: i64,
me: i64,
) -> DbResult<Vec<(String, Option<String>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT u.name, p.display_name FROM users u JOIN principals p ON p.user_id = u.id
WHERE u.active = 1 AND u.id != ?2
@@ -1980,7 +2011,7 @@ impl Db {
user_id: i64,
mode: PimShareMode,
) -> DbResult<()> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
c.execute(
"INSERT INTO pim_shares (collection_id, user_id, mode) VALUES (?1, ?2, ?3)
ON CONFLICT (collection_id, user_id) DO UPDATE SET mode = ?3",
@@ -1990,7 +2021,7 @@ impl Db {
}
pub async fn pim_remove_share(&self, collection_id: i64, user_id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute(
"DELETE FROM pim_shares WHERE collection_id = ?1 AND user_id = ?2",
params![collection_id, user_id],
@@ -1998,7 +2029,7 @@ impl Db {
}
pub async fn rooms(&self) -> DbResult<Vec<PimPrincipal>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PRINCIPAL_COLS} FROM {PRINCIPALS} WHERE p.user_id IS NULL ORDER BY p.name"
))?;
@@ -2013,7 +2044,7 @@ impl Db {
display_name: &str,
kind: UserType,
) -> DbResult<Option<PimPrincipal>> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
let inserted = tx.execute(
"INSERT INTO principals (kind, name, display_name)
@@ -2040,7 +2071,7 @@ impl Db {
}
pub async fn set_room_display_name(&self, id: i64, display_name: &str) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute(
"UPDATE principals SET display_name = ?2 WHERE id = ?1 AND user_id IS NULL",
params![id, display_name],
@@ -2049,7 +2080,7 @@ impl Db {
/// Deletes a room or resource after `ops`, as [`Self::delete_user`] does.
pub async fn delete_room(&self, id: i64, ops: &[PimOp]) -> DbResult<bool> {
let mut c = self.0.lock().await;
let mut c = self.conn.lock().await;
let tx = c.transaction()?;
apply_ops(&tx, ops)?;
let deleted = tx.execute(
@@ -2057,12 +2088,13 @@ impl Db {
[id],
)? > 0;
tx.commit()?;
self.forget_defaults();
Ok(deleted)
}
/// A principal by id, a disabled account's too.
pub async fn pim_principal_by_id(&self, id: i64) -> DbResult<Option<PimPrincipal>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(&format!(
"SELECT {PRINCIPAL_COLS} FROM {PRINCIPALS} WHERE p.id = ?1"
))?;
@@ -2077,7 +2109,7 @@ impl Db {
principal_id: i64,
needles: &[&str],
) -> DbResult<Vec<(i64, PimObject, Vec<u8>)>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let any: Vec<String> = (0..needles.len())
.map(|i| format!("instr(lower(CAST(o.data AS TEXT)), ?{}) > 0", i + 2))
.collect();
@@ -2099,7 +2131,7 @@ impl Db {
/// Every public feed link with its collection and owner, for the admin.
pub async fn all_pim_links(&self) -> DbResult<Vec<PimLinkWithOwner>> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
let mut stmt = c.prepare_cached(
"SELECT l.id, l.token, l.collection_id, l.busy_only, l.created_at, l.expires_at,
l.password_hash, coalesce(c.displayname, c.slug), c.kind, u.id, u.name,
@@ -2125,7 +2157,7 @@ impl Db {
/// Revokes a feed link whoever made it.
pub async fn admin_delete_pim_link(&self, id: i64) -> DbResult<bool> {
let c = self.0.lock().await;
let c = self.conn.lock().await;
Ok(c.execute("DELETE FROM pim_links WHERE id = ?1", [id])? > 0)
}
@@ -2605,7 +2637,8 @@ mod tests {
/// Backdate a stamp, which production code has no reason to do.
async fn set_last_used(db: &Db, secret_hash: &str, at: &str) {
db.0.lock()
db.conn
.lock()
.await
.execute(
"UPDATE app_passwords SET last_used_at = ?1 WHERE secret_hash = ?2",
@@ -2769,6 +2802,25 @@ mod tests {
);
}
#[tokio::test]
async fn deleted_defaults_come_back() {
let (db, admin) = db_with_admin().await;
let p = db.principal_of(admin.id).await.unwrap();
db.pim_ensure_defaults(p).await.unwrap();
let books = db.pim_collections(p, PimKind::AddressBook).await.unwrap();
assert_eq!(books.len(), 1);
db.pim_delete_collection(books[0].id).await.unwrap();
// Remembered defaults must not hide that one is gone.
db.pim_ensure_defaults(p).await.unwrap();
assert_eq!(
db.pim_collections(p, PimKind::AddressBook)
.await
.unwrap()
.len(),
1
);
}
#[tokio::test]
async fn rooms_are_principals_not_accounts() {
let (db, admin) = db_with_admin().await;
@@ -3116,7 +3168,7 @@ mod tests {
let stale = db.create_share_unlock(future.id).await.unwrap();
let doomed = db.create_share_unlock(past_offset.id).await.unwrap();
{
let c = db.0.lock().await;
let c = db.conn.lock().await;
c.execute(
"UPDATE share_unlocks SET created_at = '2000-01-01T00:00:00Z' WHERE token = ?1",
[&stale],
Mserver/src/main.rs
@@ -1,3 +1,6 @@
#[global_allocator]
static ALLOC: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
server::run().await
Mserver/tests/api_pim.rs
@@ -850,6 +850,27 @@ async fn calendar_reports() {
assert_eq!(error_condition(&r), Name::new(DAV, "supported-report"));
}
#[tokio::test]
async fn expand_answers_are_capped() {
let (env, auth) = setup().await;
for i in 0..4 {
let hourly = format!(
"BEGIN:VEVENT\r\nUID:h{i}\r\nDTSTAMP:20260101T000000Z\r\nDTSTART:20260101T000000Z\r\n\
DURATION:PT10M\r\nRRULE:FREQ=HOURLY\r\nSUMMARY:tick\r\nEND:VEVENT\r\n"
);
put(&env, &auth, &format!("{CAL}h{i}.ics"), &ics(&hourly)).await;
}
// 7200 instances each: three together pass the limit of one answer, one
// alone stays under that of one object. The fourth is cut off.
let expand = r#"<c:calendar-data><c:expand start="20260101T000000Z" end="20261028T000000Z"/></c:calendar-data>"#;
let range = r#"<c:comp-filter name="VEVENT"><c:time-range start="20260101T000000Z" end="20261028T000000Z"/></c:comp-filter>"#;
let r = req(&env, "REPORT", CAL, &auth, &[], &query(range, expand)).await;
assert_eq!(r.status, StatusCode::MULTI_STATUS);
let s = statuses(&r);
assert!(s.contains(&(CAL.to_string(), Some(507))), "{s:?}");
assert_eq!(s.len(), 4, "{s:?}");
}
#[tokio::test]
async fn sync_collection() {
let (env, auth) = setup().await;