Deployment
LocalS3 runs as a Docker image, as an executable jar, or embedded in a JVM (see embedding.md). All of them are configured by the same environment variables, and answer the same health check and admin endpoints.
Docker
The images are published to DockerHub: linuxchina/local-s3, for
linux/amd64 and linux/arm64, runs on a Java 25 JRE, and linuxchina/local-s3:native-<version> is a much smaller
GraalVM native image.
docker run --name s3 -d -v "$PWD/local-s3:/data" -p 29090:29090 linuxchina/local-s3
The image sets LOCAL_S3_HOST=0.0.0.0, LOCAL_S3_MODE=PERSISTENCE and AWS_BUCKETS=my-bucket, so it listens on
every interface, persists to the /data volume, and has the bucket my-bucket ready on startup. It declares a Docker HEALTHCHECK that requests
the health check.
It runs as the unprivileged user locals3, so a bind-mounted data directory must be writable by that user. To run as
your own user instead, and keep the ownership of the directory, start the container with
--user "$(id -u):$(id -g)".
A published port without credentials is open to the network. The container listens on every interface, so
-p 29090:29090lets everyone who reaches the machine read, write and delete every bucket, and the service says so when it starts:!! LocalS3 is listening on 0.0.0.0:29090 without authentication: everyone who reaches this port can read, write and delete every bucket.!! Any signature is accepted, and the admin endpoints (/_admin/ui, /_admin/reset, ...) and the Iceberg REST catalog answer anonymous requests.!! Set LOCAL_S3_ACCESS_KEY_ID and LOCAL_S3_SECRET_ACCESS_KEY, or LocalS3Builder.credentials(...), to require signed requests; bind 127.0.0.1 to serve this machine alone.On a shared network, e.g. an office network or a CI machine, set credentials, or publish the port to the loopback address of the host alone:
docker run -d -p 29090:29090 -e LOCAL_S3_ACCESS_KEY_ID=local -e LOCAL_S3_SECRET_ACCESS_KEY=local linuxchina/local-s3docker run -d -p 127.0.0.1:29090:29090 linuxchina/local-s3The warning is logged by
com.robothy.s3.rest.LocalS3; a service that is meant to be open can silence it there.
Credentials don't protect a catalog that vends them. The Iceberg REST catalog answers anonymous requests, and with credential vending on, which is its default, it hands the key pair of the service to whoever loads a table, and signs any S3 request at
/iceberg/v1/aws/s3/sign.GET /iceberg/v1/configcarries the endpoint, the region and the path-style addressing alone, but anyone who reaches the port may create a table and load it. A service with credentials, a catalog that vends them and an address other than a loopback one says so when it starts:!! The Iceberg REST catalog on 0.0.0.0:29090 answers anonymous requests and vends the credentials of this service: everyone who reaches this port can obtain the secret key from a loaded table, and have any S3 request signed at /iceberg/v1/aws/s3/sign.On a shared network, turn credential vending off,
icebergCatalog(iceberg -> iceberg.credentialVending(false)), and configure the engines with the keys by hand, or publish the port to the loopback address of the host alone.
local-s3-standalone/docker-compose.yaml starts both images side by side, on ports 29090 and 39090.
examples/lakehouse is a Compose file of LocalS3 with its Iceberg REST catalog on,
and Spark and Trino that use it.
JVM options
The JVM based image starts in about half the time that a plain java -jar takes, thanks to an
AOT cache, /app/app.aot, that the image build creates by running the service the way
the image runs it, in PERSISTENCE mode and with credentials, and sending it the common S3, S3 Vectors and Iceberg
requests, signed with SigV4 like the requests of the AWS SDKs, DuckDB and s5cmd, including bodies larger than 4 MiB,
aws-chunked bodies and a presigned URL. The cache holds the classes that the service loads, already loaded and
linked, and the profiles of their methods, so that the JIT compiles the hot ones sooner.
The image always starts with -XX:AOTCache=/app/app.aot -XX:+UseCompactObjectHeaders, whatever JAVA_OPTS says.
Compact object headers take 4 bytes less per object, which adds up for the many small
objects of the metadata, and the cache was created with them: a JVM only uses a cache that was created with the same
setting. A JVM that can't use the cache warns Unable to use AOT cache and starts without it, in about twice the time.
This happens with another jar mounted over /app/s3.jar, -XX:-UseCompactObjectHeaders or -XX:+UseZGC in
JAVA_OPTS. Other GCs, e.g. -XX:+UseG1GC, and the options below use the cache.
JAVA_OPTS defaults to -XX:MaxRAMPercentage=75.0 -XX:+UseSerialGC: the heap is sized by the memory limit of the
container, and the Serial GC takes less memory and CPU than G1 for the heaps below a few GB that a test or development
service has. Set -XX:+UseG1GC in JAVA_OPTS for a service with a large heap.
A short-lived container, e.g. one per test class in Testcontainers, starts faster and takes less memory still with a minimal configuration for tests:
docker run -d -p 29090:29090 -e LOCAL_S3_MODE=IN_MEMORY \
-e JAVA_OPTS="-Xss256k -XX:+UseSerialGC -XX:MaxRAMPercentage=50 -XX:TieredStopAtLevel=1" linuxchina/local-s3
new LocalS3Container("latest")
.withMode(LocalS3Container.Mode.IN_MEMORY)
.withEnv("JAVA_OPTS", "-Xss256k -XX:+UseSerialGC -XX:MaxRAMPercentage=50 -XX:TieredStopAtLevel=1");
| Option | Effect |
|---|---|
-Xss256k | Smaller stacks of the platform threads. The requests run on virtual threads, whose stacks grow on the heap. |
-XX:+UseSerialGC | The GC that takes the least memory and CPU for a small heap. |
-XX:MaxRAMPercentage=50 | Leaves half the memory limit of the container to the JVM beyond its heap and to the page cache. |
-XX:TieredStopAtLevel=1 | Compiles with C1 alone, which warms up faster; the peak throughput is lower, so leave it out for large uploads or benchmarks. |
Netty without sun.misc.Unsafe
On Java 25, Netty 4.2 no longer uses sun.misc.Unsafe by default, and logs this only at DEBUG level. Its buffers
use the ByteBuffer and memory segment APIs instead. LocalS3 doesn't need it either. The content of a stored file is
sent with zero-copy FileRegions, and a large request body is memory-mapped and unmapped with the FFM API, so this
fallback only affects small buffers. In our runs of small-object uploads and large GETs, the two settings showed no
difference beyond run-to-run noise.
LocalS3 therefore runs with the JVM default and doesn't ask for --sun-misc-unsafe-memory-access=allow, which goes
against JEP 471 and JEP 498: the memory-access
methods of Unsafe are deprecated for removal. To compare
throughput, add it to JAVA_OPTS or to the JVM that embeds LocalS3. The flag applies to the whole JVM, including the
host application. -Dio.netty.noUnsafe=false also turns Unsafe back on for Netty, but then the JVM prints the JEP 498
warning about the memory-access methods of Unsafe. With DEBUG logging for
io.netty.util.internal.PlatformDependent0, Netty logs which one it picked, e.g.
sun.misc.Unsafe: unavailable (io.netty.noUnsafe=true by default on Java 25+).
Executable jar
The same service runs without Docker. local-s3-standalone is published to Maven Central as an executable
jar that carries everything it needs, so a JRE 25 is the only requirement:
curl -LO https://repo1.maven.org/maven2/io/github/robothy/local-s3-standalone/2.5.0/local-s3-standalone-2.5.0.jar
java -jar local-s3-standalone-2.5.0.jar
It takes an option per setting, one for each variable below; --help lists them all, beside the
variable each one stands for:
java -jar local-s3-standalone-2.5.0.jar --port 29292 --bucket my-bucket
java -jar local-s3-standalone-2.5.0.jar --help
An option wins over the variable of that setting, which in turn wins over the system property of the same name, so the three configure the same service:
LOCAL_S3_MODE=PERSISTENCE java -jar local-s3-standalone-2.5.0.jar --data-path "$HOME/local-s3"
java -DLOCAL_S3_PORT=29090 -DLOCAL_S3_MODE=IN_MEMORY -DAWS_BUCKETS=my-bucket \
-jar local-s3-standalone-2.5.0.jar
An argument that isn't an option of the service, e.g. a misspelled --ports, an option given twice, or a value that
isn't valid, stops the jar with the reason and exit code 2, rather than starting a service that isn't the one that
was asked for.
Unlike the container, the jar listens on 127.0.0.1 and runs IN_MEMORY by default, starting from the initial data of
/data if that directory exists. Pass --host 0.0.0.0 to serve other hosts, and --mode PERSISTENCE with a
--data-path of your own to keep the data:
java -jar local-s3-standalone-2.5.0.jar --mode PERSISTENCE --data-path "$HOME/local-s3"
A shell that a developer works in often exports AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, the client
credentials of the AWS SDKs, e.g. a real key of a company account. The jar and an embedded service don't take those
for their own credentials: they read LOCAL_S3_ACCESS_KEY_ID and LOCAL_S3_SECRET_ACCESS_KEY (or --access-key and
--secret-key), and AWS_* only with LOCAL_S3_CREDENTIALS_FROM_AWS_ENV=true. The Docker images set that flag, since
a container doesn't inherit the shell of a developer, so -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... keeps
working there. When credentials are configured, the startup log names the variables they came from and the access key
ID, masked:
LocalS3 credentials from the variables LOCAL_S3_ACCESS_KEY_ID and LOCAL_S3_SECRET_ACCESS_KEY: access key ID AKIA****1234.
Configuration
Every variable of the table below is also an option of the executable jar, named after it:
LOCAL_S3_PORT is --port, LOCAL_S3_WEBSITE_ALL_BUCKETS is --website-all-buckets, AWS_BUCKETS is --buckets
(or --bucket, repeated), and LOCAL_S3_ACCESS_KEY_ID and LOCAL_S3_SECRET_ACCESS_KEY are --access-key and --secret-key.
The exceptions are the cache limits, which are read when a cache is created rather than applied to a service, and the
logging variables, which the logging configuration of the jar and the image reads; they are set by a variable alone. A container is configured by the variables; java -jar s3.jar --help prints the options.
| Variable | Default | Description |
|---|---|---|
LOCAL_S3_PORT | 29090 | Port that LocalS3 listens on. |
LOCAL_S3_HOST | 127.0.0.1; 0.0.0.0 in the image | Host to bind. |
LOCAL_S3_MODE | IN_MEMORY; PERSISTENCE in the image | PERSISTENCE or IN_MEMORY. |
LOCAL_S3_DATA_PATH | /data | Data directory, or initial data in IN_MEMORY mode. |
LOCAL_S3_IN_MEMORY_MAX_BYTES | a quarter of the max heap | IN_MEMORY mode: the max heap that the objects and parts stored in the service take, e.g. 512m. An upload beyond it is answered with 507 InsufficientStorage; delete objects, raise the limit, or use PERSISTENCE mode for large data. The initial data of the data path doesn't count. |
LOCAL_S3_PERSISTENCE_POLICY | DURABLE | PERSISTENCE mode: when changes are written to the file. DURABLE commits every change, so a killed process loses nothing (it doesn't fsync, so a power loss may); FAST commits in the background and on shutdown, which is much quicker and writes far less. See Persistence policy. |
LOCAL_S3_LIFECYCLE_INTERVAL | 0, off | Apply the lifecycle configurations of the buckets on a schedule: once the service has started, then every interval, e.g. 1h, 30m or PT1H. For a service that runs for a long time, so that what the rules expire doesn't pile up; a test applies them with POST /_admin/lifecycle instead. |
LOCAL_S3_COMPOSITE_MULTIPART_ETAGS | true | Give the object of a completed multipart upload the entity tag of Amazon S3, i.e. the digest of the digests of its parts with a -<parts> suffix. false answers the digest of the whole content, which LocalS3 answered before 2.5. |
LOCAL_S3_ACCEPT_CHUNKED_UPLOADS | true | Store a PutObject or UploadPart body sent with Transfer-Encoding: chunked and no Content-Length, e.g. of curl -T -. false answers 411 MissingContentLength, like Amazon S3. See request validation. |
LOCAL_S3_VIRTUAL_HOST_DOMAINS | Comma-separated base domains of virtual-hosted-style requests, e.g. s3,s3.local for my-bucket.s3. localhost, 127-0-0-1.nip.io (see Virtual-hosted-style requests), Amazon S3 (*.amazonaws.com), Alibaba Cloud OSS (my-bucket.oss-cn-hangzhou.aliyuncs.com), Cloudflare R2 (my-bucket.<account-id>.r2.cloudflarestorage.com) and Tigris (my-bucket.t3.storage.dev, my-bucket.fly.storage.tigris.dev) hosts always work. | |
LOCAL_S3_VIRTUAL_THREADS | true | Handle every request on a virtual thread of its own. false handles the requests on a pool of platform threads, as many as the machine has processors and at least 4. |
LOCAL_S3_OBJECT_METADATA_CACHE_MAX_ENTRIES | 50000 | PERSISTENCE mode: the number of objects whose metadata is kept in heap; see Opening a large data path. |
LOCAL_S3_INITIAL_DATA_CACHE_MAX_ENTRIES | 1024 | IN_MEMORY mode with initial data: the max number of data paths whose loaded data the JVM caches. |
LOCAL_S3_INITIAL_DATA_CACHE_MAX_BYTES | a quarter of the max heap | IN_MEMORY mode with initial data: the max heap that the copies of the objects read from the data paths take, e.g. 512m. The least recently used data paths are dropped to make room, and an object that still doesn't fit is read from the disk instead. Also settable with LocalS3.configureInitialDataCache(maxEntries, maxBytes). |
AWS_BUCKETS | none; my-bucket in the image | Comma-separated buckets to create on startup. A name with the suffix :versioned, e.g. plain,audit:versioned, is created with versioning enabled. |
LOCAL_S3_ACCESS_KEY_ID, LOCAL_S3_SECRET_ACCESS_KEY | Require requests signed with this key pair. Unset, every request is answered, whoever sends it; a service that also binds an address other than a loopback one warns about that when it starts, see Docker. | |
LOCAL_S3_CREDENTIALS_FROM_AWS_ENV | false; true in the image | Read the key pair from AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY where LOCAL_S3_ACCESS_KEY_ID isn't set. Off outside the image, because those are the client credentials that a developer's shell exports; see Executable jar. |
LOCAL_S3_ALLOWED_CLOCK_SKEW | 15m; 0 in the Docker images | With credentials: how far the time of a signed request may be from the clock of the service, e.g. 1h or PT1H, before it is answered with 403 RequestTimeTooSkewed. 0 turns the time check off, e.g. for a VM whose clock drifts after the host sleeps, and with it the expiry of presigned URLs (X-Amz-Expires): an expired URL is still answered. The Docker images set 0, so a test of an expired presigned URL against a container sets a duration, e.g. -e LOCAL_S3_ALLOWED_CLOCK_SKEW=15m. See request validation. |
LOCAL_S3_FEATURES_VECTOR, LOCAL_S3_FEATURES_S3_TABLES, LOCAL_S3_FEATURES_KMS, LOCAL_S3_FEATURES_STS, LOCAL_S3_FEATURES_CONSOLE | true | false leaves the routes of the S3 Vectors API, the S3 Tables API, AWS KMS, AWS STS or the console out of the service. See Turn features off. |
LOCAL_S3_FEATURES_MCP | false | true serves the MCP server at /_mcp, for AI agents. |
LOCAL_S3_WEBSITE | true | Serve the buckets as static websites to the requests that carry no credentials. Only a public bucket answers one; the signed requests of an S3 client are never affected. |
LOCAL_S3_WEBSITE_ALL_BUCKETS | false | Serve every bucket as a static website, not the public ones alone, which also lets an unsigned request read the objects of a private bucket. Meant for local development. |
LOCAL_S3_WEBSITE_INDEX_DOCUMENT | index.html | The index document of the buckets that have no WebsiteConfiguration of their own. |
LOCAL_S3_WEBSITE_ERROR_DOCUMENT | The error document of the buckets that have no WebsiteConfiguration of their own, e.g. error.html; unset answers a generic error page. | |
LOCAL_S3_CORS_ALLOWED_ORIGINS | Comma-separated origins that the default CORS rule allows, e.g. * or http://localhost:5173. Setting it turns the rule on; it applies to the buckets that have no CORS configuration of their own and to the requests that address no bucket. * lets every web page open in a browser reach the service. | |
LOCAL_S3_CORS_ALLOWED_METHODS | GET,PUT,POST,DELETE,HEAD | Comma-separated methods that the default CORS rule allows. |
LOCAL_S3_CORS_ALLOWED_HEADERS | * | Comma-separated request headers that the default CORS rule allows. |
LOCAL_S3_CORS_EXPOSE_HEADERS | ETag, x-amz-version-id, … | Comma-separated response headers that the default CORS rule lets a page read. |
LOCAL_S3_CORS_MAX_AGE_SECONDS | Seconds that a browser may cache a preflight response of the default CORS rule. | |
LOCAL_S3_TLS_CERT, LOCAL_S3_TLS_KEY | Serve HTTPS, alongside plain HTTP on the same port, with this certificate chain and unencrypted PKCS#8 private key, each the path of a PEM file or the PEM content itself. Set both or neither; see HTTPS. | |
LOCAL_S3_TLS_SELF_SIGNED | Serve HTTPS with a certificate that the service generates for itself on startup: true issues it for localhost, 127.0.0.1 and ::1, and a comma-separated list of hosts issues it for those. Not to be set together with LOCAL_S3_TLS_CERT; see Generate a certificate on startup. Issued by the CA of mkcert where mkcert is installed; see LocalS3 with mkcert. | |
LOCAL_S3_TLS_REQUIRED | false | Serve HTTPS alone, instead of answering HTTP and HTTPS on the same port, so that a plain HTTP request fails. No effect without a certificate; see HTTPS. |
LOCAL_S3_LOGGING_LEVEL | INFO | The level of the log of the jar and the image, e.g. DEBUG. |
LOCAL_S3_LOG_COLOR | false | true colors the log for a terminal. Off by default, so that docker logs, CI logs and log files hold no escape sequences. The log is timestamped in the time zone of the process, with its offset, e.g. 2026-09-30T13:31:01.319+02:00; set TZ to change it, e.g. TZ=UTC for a container. |
JAVA_OPTS | -XX:MaxRAMPercentage=75.0 -XX:+UseSerialGC | JVM options of the JVM based image; see JVM options. |
The same variables configure an embedded service, through LocalS3Builder.fromEnvironment(), which reads them
from the environment or from the system properties of the same names. Only the variables that are set are
applied, so the defaults of the embedded service are kept for the rest; the defaults in the table above are
those of the jar and the image. An embedded service has no data path unless one is configured. The names of the
variables are the constants of LocalS3Environment.
LocalS3 localS3 = LocalS3.builder()
.port(29090)
.fromEnvironment()
.build();
Virtual-hosted-style requests
A virtual-hosted-style request names the bucket in the host, e.g. my-bucket.localhost:29090, rather than in the path.
LocalS3 takes the bucket from the host when the host is a subdomain of a base domain: localhost, 127.0.0.1,
0.0.0.0 and 127-0-0-1.nip.io always are, besides those of LOCAL_S3_VIRTUAL_HOST_DOMAINS. A request to a base
domain itself is path-style.
Amazon S3 hosts work too, e.g. for a client whose /etc/hosts points them to LocalS3. The domain is matched
case-insensitively and the bucket keeps its case:
| Host | Bucket region |
|---|---|
my-bucket.s3.us-west-2.amazonaws.com, my-bucket.s3.dualstack.us-west-2.amazonaws.com, my-bucket.s3-fips.us-west-2.amazonaws.com | us-west-2 |
my-bucket.s3-us-west-2.amazonaws.com (legacy dash-region) | us-west-2 |
my-bucket.s3.amazonaws.com, my-bucket.s3-external-1.amazonaws.com | local |
my-bucket.s3-accelerate.amazonaws.com, my-bucket.s3-accelerate.dualstack.amazonaws.com | local |
The regional endpoints themselves, e.g. s3.us-west-2.amazonaws.com or s3-us-west-2.amazonaws.com, are path-style
with that region; other AWS hosts, e.g. s3tables.us-east-1.amazonaws.com, are path-style.
The client still has to resolve the host to LocalS3. *.localhost resolves to the loopback address in browsers and
in curl, but not in every client, e.g. not in the resolver of the JVM; a configured domain needs an /etc/hosts
entry or a DNS record per bucket. nip.io is a public wildcard DNS that resolves every name under
127-0-0-1.nip.io to 127.0.0.1, so virtual-hosted-style requests work with no configuration at all:
aws s3 ls s3://my-bucket --endpoint-url http://127-0-0-1.nip.io:29090
curl http://my-bucket.127-0-0-1.nip.io:29090/hello.txt
S3Client s3 = S3Client.builder()
.endpointOverride(URI.create("http://127-0-0-1.nip.io:29090"))
.region(Region.US_EAST_1)
.build(); // virtual-hosted style by default: my-bucket.127-0-0-1.nip.io
Resolving the names needs the DNS of nip.io, i.e. network access; a DNS resolver that blocks answers with private
addresses (DNS rebinding protection, e.g. of some routers) drops them. Offline, use path-style requests or localhost.
HTTPS
LocalS3 serves plain HTTP unless it is given a certificate and its private key in PEM format. Several clients use, or require, HTTPS by default, and then connect to LocalS3 without turning TLS off:
| Client | Default that expects HTTPS |
|---|---|
DuckDB httpfs | USE_SSL true in a secret, s3_use_ssl = true |
object_store (Rust) and some Go clients | an http:// endpoint is refused unless e.g. allow_http is set |
With TLS configured, the port serves both HTTP and HTTPS: each connection is told apart by its first bytes, so a
client that speaks TLS and one that doesn't reach the same endpoint, and a test suite covering both needs one service
rather than two. The health check answers at https://…/_health and http://…/_health alike.
Set LOCAL_S3_TLS_REQUIRED=true, or tls(tls -> tls.required(true)), to serve HTTPS alone, which makes a plain HTTP request fail;
that is what a test asserting that its client really uses TLS needs. Without a certificate the setting has no effect.
There are two ways to get a certificate: LocalS3 generates one for itself, which needs nothing installed but has to be handed to every client, or mkcert issues one that the machine already trusts, which clients then need nothing for. Where mkcert is installed, the two come together: the certificate that LocalS3 generates is issued by the CA of mkcert, so the machine trusts it without a file to create.
Generate a certificate on startup
LOCAL_S3_TLS_SELF_SIGNED=true, or tls(LocalS3Tls.selfSigned()), generates a certificate for localhost,
127.0.0.1 and ::1 when the service starts, in a few milliseconds and with nothing to install:
# Executable jar
LOCAL_S3_TLS_SELF_SIGNED=true java -jar local-s3-standalone-2.5.0.jar
# Docker
docker run -d -p 29090:29090 -e LOCAL_S3_TLS_SELF_SIGNED=true linuxchina/local-s3
Set it to the comma-separated hosts to issue the certificate for instead of true, for every name that clients reach
the service by, e.g. LOCAL_S3_TLS_SELF_SIGNED=localhost,127.0.0.1,local-s3 for the service name in docker-compose, or
localhost,*.s3.local for virtual-hosted-style requests. A client verifies the host it connects to
against them, and refuses a host that the certificate doesn't name.
Where mkcert is installed, the certificate is issued by its CA instead, and the rest of this section doesn't apply; see LocalS3 with mkcert.
No client trusts the certificate, because it signed itself. The service logs where it looked for the CA of mkcert, and the certificate in PEM format when it starts, which is the only place it exists:
LocalS3 found no CA of mkcert in /Users/me/Library/Application Support/mkcert, and generates a self-signed certificate.
LocalS3 generated a self-signed certificate for this service: CN=localhost,O=LocalS3 for
[localhost, 127.0.0.1, 0:0:0:0:0:0:0:1], self-signed, valid until 2027-09-18T03:15:41Z, SHA-256 fingerprint 05:B7:…
-----BEGIN CERTIFICATE-----
MIIBtTCCAVqgAwIBAgIUBBaG2d/qt2FBS7W0dsx8AiwalSowCgYIKoZIzj0EAwIw
…
-----END CERTIFICATE-----
Save that block to a file, e.g. local-s3.pem, and hand it to the client as its CA: curl --cacert local-s3.pem,
AWS_CA_BUNDLE=local-s3.pem, SET ca_cert_file = 'local-s3.pem' in DuckDB, or the other clients of
Trust the certificate in clients. In the JVM that embeds the service,
localS3.getConfig().tls() holds it, and newClientSslContext() trusts it without a file:
LocalS3 localS3 = LocalS3.builder().port(29090).tls(LocalS3Tls.selfSigned()).build();
localS3.start();
LocalS3Tls tls = localS3.getConfig().tls();
Files.writeString(Path.of("local-s3.pem"), tls.certificateChainPem()); // for a client outside the JVM
SSLContext sslContext = tls.newClientSslContext(); // for one inside it
The certificate is valid for a year, and a new one is generated on every start, so a client that was given the certificate of one run has to be given the next one as well. Use mkcert below where that gets in the way, e.g. for a machine that several people or containers develop against.
LocalS3 with mkcert
When LocalS3 generates its certificate, with LOCAL_S3_TLS_SELF_SIGNED or LocalS3Tls.selfSigned(), it looks for the
CA of mkcert, in the directory that mkcert -CAROOT prints:
| Where | Directory |
|---|---|
CAROOT is set | $CAROOT |
| macOS | ~/Library/Application Support/mkcert |
| Linux and other Unix | $XDG_DATA_HOME/mkcert, or ~/.local/share/mkcert |
| Windows | %LOCALAPPDATA%\mkcert |
If rootCA.pem and rootCA-key.pem are there, the CA of mkcert issues the certificate rather than the certificate
signing itself. It names the same hosts and is valid for the same year, or until the CA expires if that is sooner, and
it is no CA itself. After mkcert -install, the machine trusts it like a certificate that mkcert created:
brew install mkcert # or see the installation of mkcert for Linux and Windows
mkcert -install # once: creates the CA, and adds it to the trust stores of the system and of browsers
LOCAL_S3_TLS_SELF_SIGNED=localhost,127.0.0.1,s3.local java -jar local-s3-standalone-2.5.0.jar
curl https://localhost:29090/_health # no --cacert
The service logs which CA issued the certificate, by the directory, subject and SHA-256 fingerprint of the CA, so that
a client that refuses the certificate can be checked against the CA it trusts, e.g. with
openssl x509 -noout -fingerprint -sha256 -in "$(mkcert -CAROOT)/rootCA.pem". It doesn't log the certificate in PEM
format, since the CA is the file to trust:
LocalS3 issued its certificate with the CA of mkcert in /Users/me/Library/Application Support/mkcert:
CN=mkcert me@laptop,OU=me@laptop,O=mkcert development CA, SHA-256 fingerprint 9F:21:…
LocalS3 serves HTTPS with the certificate CN=localhost,O=LocalS3 for [localhost, 127.0.0.1, s3.local], issued by
CN=mkcert me@laptop,OU=me@laptop,O=mkcert development CA, valid until 2027-10-05T08:00:00Z, SHA-256 fingerprint 3A:7B:…
Compared with a certificate that mkcert creates, there is no file to create and
pass to the service, and the hosts are chosen on every start. Clients that don't read the trust store of the system,
e.g. JVM clients and the AWS CLI, still need "$(mkcert -CAROOT)/rootCA.pem"; see
Trust the certificate in clients. In the JVM that embeds the service,
LocalS3Tls.newClientSslContext() and trustManagers() trust the certificate as before.
The certificate signs itself as before when there is no CA of mkcert, i.e. mkcert isn't installed or mkcert -install
hasn't run yet. It does too when the CA can't be used: a file can't be read, the key is not an unencrypted PKCS#8 key
or doesn't belong to the certificate, or the certificate is not a CA or has expired; the service then logs a warning
that names the directory and says why. To have the certificate sign itself where mkcert is installed, e.g. in a test that expects that, point
CAROOT at a directory without a CA, e.g. CAROOT=/nonexistent.
Docker: a container doesn't see the CA of mkcert on the host, so the image generates a certificate that signs itself. To serve HTTPS that the host trusts from a container, create the certificate with mkcert on the host, and mount it.
rootCA-key.pem issues certificates for any host that the machine trusts. LocalS3 reads it to sign its certificate and
keeps it nowhere else, neither on disk nor in the log; still, any process that reads it can intercept the HTTPS
connections of the machine, so keep the directory of mkcert readable by its owner alone, which is what mkcert creates.
Create a certificate with mkcert
mkcert creates a certificate that the machine trusts, without a warning in browsers, curl or DuckDB:
brew install mkcert # or see the installation of mkcert for Linux and Windows
mkcert -install # once: adds the CA of mkcert to the trust stores of the system and of browsers
mkcert localhost 127.0.0.1 # creates localhost+1.pem and localhost+1-key.pem in the current directory
List every name that clients reach LocalS3 by, e.g. mkcert localhost 127.0.0.1 s3 s3.local "*.s3.local" for the
service name in docker-compose and for virtual-hosted-style requests. The key that mkcert writes is
an unencrypted PKCS#8 key (-----BEGIN PRIVATE KEY-----), which is what LocalS3 expects.
Start LocalS3 with the certificate
# Executable jar
LOCAL_S3_TLS_CERT=localhost+1.pem LOCAL_S3_TLS_KEY=localhost+1-key.pem java -jar local-s3-standalone-2.5.0.jar
# Docker
docker run -d -p 29090:29090 -v "$PWD:/certs:ro" \
-e LOCAL_S3_TLS_CERT=/certs/localhost+1.pem -e LOCAL_S3_TLS_KEY=/certs/localhost+1-key.pem \
linuxchina/local-s3
curl https://localhost:29090/_health
mkcert writes the key readable by its owner only, and the image runs as the user locals3: make the key readable in
the container, e.g. chmod 644 localhost+1-key.pem for a local certificate, or start it with
--user "$(id -u):$(id -g)".
Embedded, LocalS3.builder().tls("localhost+1.pem", "localhost+1-key.pem"); see
embedding.md. The files are read when the service is configured, and an unreadable file,
or a key that doesn't belong to the certificate, fails right away.
Trust the certificate in clients
mkcert -install makes the system trust the certificate, which covers curl, DuckDB, and most Go and Rust clients.
Other clients need the CA of mkcert, "$(mkcert -CAROOT)/rootCA.pem", explicitly. A
generated certificate is trusted by nothing, so every client below needs it, with
the PEM file that the startup log was saved to in place of rootCA.pem:
- JVM (AWS SDK for Java, Hadoop S3A, Iceberg, Spark): the JVM has its own trust store. Import the CA once, e.g.
keytool -importcert -noprompt -alias mkcert -cacerts -storepass changeit -file "$(mkcert -CAROOT)/rootCA.pem", or pass a trust store with-Djavax.net.ssl.trustStore=…. - AWS CLI and boto3:
AWS_CA_BUNDLE="$(mkcert -CAROOT)/rootCA.pem", or--ca-bundle. - Python requests / Node.js:
REQUESTS_CA_BUNDLE/NODE_EXTRA_CA_CERTSset to the same file. - Containers: a client in another container doesn't see the trust store of the host; mount
rootCA.peminto it and point the client at it.
DuckDB, with the defaults of a secret, which uses HTTPS:
-- Only for a certificate that the machine doesn't trust, e.g. a generated one; mkcert needs no CA file.
SET ca_cert_file = 'local-s3.pem';
CREATE SECRET local_s3 (
TYPE s3,
ENDPOINT 'localhost:29090',
URL_STYLE 'path',
KEY_ID 'admin',
SECRET 'admin',
REGION 'us-east-1'
);
Without the ca_cert_file of a certificate it doesn't trust, DuckDB fails with
IO Error: SSL peer certificate or SSH remote key was not OK, or SSL connect error against a plain HTTP service.
Data directory
In PERSISTENCE mode LocalS3 loads data from and stores all data into its data directory. In IN_MEMORY mode with a
data directory, it starts from the data of the directory and never writes to it. How the directory is laid out is
described in architecture.md.
Persistence policy
A PERSISTENCE service commits the metadata of every change before it answers the request, by default. That is what
DURABLE means: a process that is killed, or that crashes, loses nothing it answered a request for. It is not an
fsync: a commit writes buckets.mvstore without syncing it to the disk, so the operating system holds the last
commits until it writes them back, and a power loss or an operating system crash may lose them. Neither policy protects
against that. The content of objects isn't synced either: after a power loss, an object's metadata may survive while its content is truncated or missing.
A commit appends a chunk to buckets.mvstore rather than replacing what it supersedes. The requests that change a
bucket at the same time share a commit: each commits once the lock of the bucket is released, and a commit that starts
meanwhile takes the changes of all the requests waiting for it. A client that writes one object after the other still
gets a commit each, so a bulk load grows the file far beyond the metadata it holds while it runs.
storage(storage -> storage.persistencePolicy(FAST)), LOCAL_S3_PERSISTENCE_POLICY=FAST, or
local-s3.persistence-policy=fast in Spring Boot, lets the store commit in the background instead, at most a second
after a change, and commits what is left when the service is shut down:
LocalS3 localS3 = LocalS3.builder()
.storage(storage -> storage.mode(LocalS3Mode.PERSISTENCE)
.dataPath("/data")
.persistencePolicy(PersistencePolicy.FAST))
.build();
Storing twenty thousand small objects through a service, measured end to end:
| Policy | Writers | Load | buckets.mvstore after the load | 30 s later | at rest |
|---|---|---|---|---|---|
DURABLE | 1 | 5.2 s | 630 MB | 27 MB | 2.4 MB |
DURABLE | 16 | 1.9 s | 267 MB | 27 MB | 2.4 MB |
FAST | 1 | 2.9 s | 7 MB | 10 MB | 2.3 MB |
FAST | 16 | 1.8 s | 5 MB | 10 MB | 2.4 MB |
A killed FAST process loses the changes of the last second, and nothing before them: the content that a change
replaces or deletes, e.g. the previous content of an overwritten object, is kept until the change is written, so the
metadata that a killed process leaves still finds the content it references. Content that is left unreferenced this
way is deleted the next time the data directory is opened. A process that is shut down, including by the JVM
shutdown hook, persists everything. That is the trade a data directory built to test against can usually make, e.g. the table
of a big data engine, which is built again if it is lost, or the data of a service embedded in an application or an
IDE. Use DURABLE when the data directory itself is what matters.
The defaults follow that: LocalS3Builder, the standalone jar and the Docker image use DURABLE; the Spring Boot
starter uses FAST. An IDE or any other application that embeds LocalS3 through LocalS3Builder should set FAST
itself unless its data can't be built again.
Either way the room of superseded chunks is given back:
- while the service runs: every 30 seconds, a store whose
buckets.mvstoreis at least 16 MB and less than half used by live metadata is compacted, which rewrites the live metadata, syncs it, and truncates the file. It holds the store for at most a second, so the commits of the requests in flight wait for it. A service that runs for long, e.g. in an IDE, therefore doesn't keep the room of every write it ever took; - when the last holder closes the store: the file is compacted fully, so a data directory rests at the size of the metadata it holds. Compaction costs the live metadata, not the size the file grew to, so it stays quick.
Opening a large data path
A PERSISTENCE service reads the keys of the objects of its data path when it starts, not their metadata: the
metadata of an object is read from buckets.mvstore when a request needs it, and a bounded number of them is kept in
heap. Opening a data path therefore costs the keys it holds rather than every version, tag and ACL of every object,
which is what lets a directory of a few hundred thousand data files, e.g. an Iceberg table, be served without loading
it all first.
The keys themselves stay in heap, one sorted map per bucket, whatever the size of the data path. Measured with a
bucket of one million objects written by a PERSISTENCE service, as the heap of the service it was opened by again, after
a full GC and before any object was read:
| Keys | Key length | Heap for 1,000,000 keys | Per key |
|---|---|---|---|
key-00000001 | 12 chars | ~170 MB | ~180 B |
warehouse/db/events/data/event_date=2026-09-01/00000-5-3f2a9c1e-….parquet (Iceberg-like) | ~100 chars | ~250 MB | ~260 B |
That is roughly 170 bytes per key plus one byte per character of an ASCII key (two per character for a key with
characters beyond Latin-1), or 200–300 MB of heap per million keys for typical keys. The metadata of the objects
that are read comes on top, bounded as described next; give a JVM that opens a data path of several million keys a
-Xmx that leaves room for both.
GET /_admin/stats reports the boundary, so it can be seen rather than guessed:
{
"data": {
"objects": 412000,
"loadedObjects": 50000,
"loadedObjectMetadataBytes": 18432000
}
}
loadedObjects is how many objects have their metadata in heap and loadedObjectMetadataBytes an estimate of what
that costs, measured as the size of the persisted form of that metadata; an IN_MEMORY service never writes its metadata, so
it reports 0. The bound defaults to 50000 objects and is
configured with the environment variable, or system property, LOCAL_S3_OBJECT_METADATA_CACHE_MAX_ENTRIES:
LOCAL_S3_OBJECT_METADATA_CACHE_MAX_ENTRIES=200000 java -jar local-s3-standalone-2.5.0.jar
Raising it trades heap for fewer reads of the store; the objects that a service serves stay in heap either way, since
what is dropped is what was read longest ago. An IN_MEMORY service holds all of its objects, having nowhere to read
them back from, so the bound doesn't apply to it and loadedObjects equals objects.
Health check
LocalS3 answers GET /_health (and HEAD /_health) with 200 OK and {"status":"UP"} once it serves
requests. The health check needs no authentication, even if credentials are configured, so probes can use it.
// Testcontainers: wait for the health check instead of the startup log message.
new LocalS3Container("latest")
.withRandomHttpPort()
.waitingFor(Wait.forHttp("/_health").forPort(29090));
LocalS3Container still waits for the startup log message by default, so that it works with images older than
the health check.
Kubernetes
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-s3
spec:
replicas: 1
selector:
matchLabels:
app: local-s3
template:
metadata:
labels:
app: local-s3
spec:
containers:
- name: local-s3
image: linuxchina/local-s3:latest
env:
- name: LOCAL_S3_MODE
value: IN_MEMORY
- name: AWS_BUCKETS
value: my-bucket
ports:
- containerPort: 29090
readinessProbe:
httpGet:
path: /_health
port: 29090
livenessProbe:
httpGet:
path: /_health
port: 29090
Run a single replica: a data directory is served by one process at a time, and the replicas of an IN_MEMORY
deployment would each hold data of their own.
Console
LocalS3 serves a built-in console at GET /_admin/ui: a single self-contained HTML page, with no build step and no
frontend framework, that lists the buckets and creates one, walks the objects of a bucket by their prefixes, previews
or downloads what is in them, shows their metadata, tags and versions, and uploads or deletes an object. Its Tables
and Vectors views browse the Iceberg catalogs and the vector buckets of the service. Open
http://localhost:29090/_admin/ui in a browser.
It answers the question a mock service otherwise leaves to aws s3 ls: what is actually in there? That matters when
LocalS3 is embedded in an IDE or an application, and when an AI agent stores artifacts in it — a person then reads the
charts, reports and datasets the agent produced instead of listing keys.
| What it does | |
|---|---|
| Buckets | Name, region and creation date, with a filter. + NEW creates one, in the default region and under the naming rules of Amazon S3, and opens it. |
| Objects | The objects and the directories under a prefix, with size and last-modified time, a prefix filter, and a page of 200 at a time. |
| An object | A preview of text, JSON, CSV, Markdown, images, audio, video, PDF and HTML, the rest as a download; Download saves it, and Share, on the row or in the preview, copies a presigned URL of the object that reads it without credentials, valid for the time chosen next to Upload (15 minutes to 7 days, an hour by default); a service without credentials copies the plain URL instead, which never expires. |
| Upload | Drop files or folders anywhere on the listing, or use Upload: each file is stored under the prefix that is open, a dropped folder becomes a prefix, and a panel shows the progress of the batch, three files at a time. A file that the browser knows no type for is stored with the content type of its extension. |
| Properties | The Properties tab of the preview: size, ETag, content type, storage class, checksum, the encryption and the Object Lock retention and legal hold recorded with it, the system metadata it is served with (Cache-Control, Content-Disposition and the rest), its x-amz-meta-* user metadata, and its tags. |
| Versions | The Versions tab of the preview: the versioning of the bucket, and the versions and the delete markers of the key, newest first (at most 1000). Open previews a version, with its own properties, Download saves it, and Delete deletes that version for good, after a confirmation. A bucket that was never versioned has the one version, null. |
| Delete | Delete on a row, after a confirmation. In a versioned bucket it puts a delete marker, like DeleteObject does. |
| Tables | Shown when the service serves the Iceberg REST catalog or table buckets. The sidebar lists the catalogs: the one at /iceberg, and one per table bucket. A catalog is walked namespace by namespace, down to its tables and views; a table shows its record count, data files and size, its location, with Browse files that opens it in the objects view, Copy DuckDB query that copies an iceberg_scan of its metadata file, and tabs of its snapshots (the current one and the branches and tags that point at each, with its summary and manifest list on a click), its current schema, its partition spec and sort order, its properties, and its metadata as JSON. |
| Vectors | Shown when the service serves the S3 Vectors API. The sidebar lists the vector buckets; a bucket lists its indexes with their dimension, distance metric, data type, number of vectors and non-filterable metadata keys; an index lists its vectors with their metadata, 100 at a time, and Show shows the data of one, with its L2 norm, and copies it as JSON. |
| Connect | Connect shows what DuckDB, s5cmd, the AWS CLI, boto3, Polars, PyIceberg and Spark need to reach the service, written by the service itself, with a Copy button: the connection snippets of the bucket or the folder that is open. DuckDB SQL in the preview of an object shows the DuckDB script that ends with a query of that object, e.g. SELECT * FROM 's3://lake/events.parquet' LIMIT 10;, which Copy query copies alone. |
Behind the page are endpoints of its own, which call the same services the S3 operations do:
GET /_admin/ui/buckets, GET /_admin/ui/objects?bucket=&prefix=&continuation-token=,
GET /_admin/ui/object?bucket=&key=, PUT /_admin/ui/object?bucket=&key=,
DELETE /_admin/ui/object?bucket=&key=, PUT /_admin/ui/bucket?bucket=, GET /_admin/ui/snippets?bucket=&key=,
GET /_admin/ui/presign?bucket=&key=&expires=<seconds>&method=GET|PUT,
GET /_admin/ui/metadata?bucket=&key=&versionId=, GET /_admin/ui/versions?bucket=&key=, GET /_admin/ui/object and
DELETE /_admin/ui/object with a versionId= for a version, GET /_admin/ui/features, which says whether the
Tables and Vectors views are shown, GET /_admin/ui/catalogs, GET /_admin/ui/namespace?tableBucket=&namespace=,
GET /_admin/ui/table?tableBucket=&namespace=&table=, where no tableBucket is the catalog at /iceberg and the
levels of a namespace are separated by %1F like in the Iceberg REST API, GET /_admin/ui/vector-buckets,
GET /_admin/ui/vector-indexes?vectorBucket=, GET /_admin/ui/vectors?vectorBucket=&index=&nextToken= and
GET /_admin/ui/vector?vectorBucket=&index=&key=,
and the multipart upload of a large file: POST /_admin/ui/multipart?bucket=&key= starts it and answers its
uploadId, PUT /_admin/ui/multipart?bucket=&key=&uploadId=&partNumber= stores a part,
POST /_admin/ui/multipart/complete?bucket=&key=&uploadId= completes it with {"parts": [{"partNumber": 1, "etag": "..."}]}, and DELETE /_admin/ui/multipart?bucket=&key=&uploadId= aborts it. They answer JSON, or the content of
the object.
A file of up to 64 MiB is uploaded with a single PutObject; a larger one is uploaded in parts of 16 MiB, three
at a time, like an S3 client would, so a large dataset dragged onto the page is stored even when it is larger than
maxRequestBodySize, and a failed upload is aborted. The console deletes no bucket and
changes no bucket configuration — versioning, policies, CORS and the rest: for those, use the S3 API. It changes no
metadata or tags of an object either, and the Tables and Vectors views only read: the catalogs and the vector
buckets are changed by the clients that own them. The table endpoint answers the IDs of snapshots as strings rather
than numbers, which JavaScript would round.
A browser can't sign a request with AWS Signature Version 4, so the console is guarded with HTTP Basic authentication rather than a signature: a service configured with credentials asks for the access key ID as the user name and the secret access key as the password, and a service without credentials, which answers unsigned S3 requests anyway, serves the console to everyone who reaches the port. The S3 API of the same service is unaffected: it keeps requiring signatures.
The endpoints that change the data — creating a bucket, the PUT and DELETE of an object, and the multipart
endpoints — also want the X-LocalS3-Console header, which the page sends and which a browser lets no other origin
send without this service allowing it first. The service never does: a CORS preflight of a path under /_admin/ui is
answered 403, even when the default CORS rule allows every header. So a page a
user has open elsewhere can't create, upload or delete through the console, whether the service has credentials or not.
curl -s -u "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" "http://localhost:29090/_admin/ui/objects?bucket=my-bucket"
curl -s -u "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" -H "X-LocalS3-Console: 1" \
-X PUT "http://localhost:29090/_admin/ui/bucket?bucket=my-bucket"
curl -s -u "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" -H "X-LocalS3-Console: 1" \
-X PUT --data-binary @report.html "http://localhost:29090/_admin/ui/object?bucket=my-bucket&key=report.html"
The content of an object is served with Content-Security-Policy: sandbox, so a stored HTML page is previewed in a
sandbox of its own rather than as a page of the console; an object stored without a content type is previewed as the
type of its extension, the way static website hosting serves one. The requests
of the console aren't recorded in the statistics of the service, so browsing doesn't show up as traffic of the
application under test; the bucket it creates and what it uploads and deletes do reach the
change listeners, like every other change.
Being paths of the service, /_admin/ui and the paths below it are the console rather than the objects of a bucket
named _admin, like the other /_admin endpoints and /_health are.
MCP server
LocalS3 serves a built-in Model Context Protocol server at /_mcp, so that an AI
agent — Claude Code, an IDE agent, or any MCP client — keeps the artifacts it produces in LocalS3 and hands them to a
person, where the console shows them. It is off by default: turn it on with
LOCAL_S3_FEATURES_MCP=true, --features-mcp, features(f -> f.mcp(true)) or local-s3.features.mcp=true.
It speaks the Streamable HTTP transport, protocol versions 2025-11-25, 2025-06-18, 2025-03-26 and 2024-11-05,
with JSON-RPC over the JSON library LocalS3 already has, so it adds no dependency. Every request is answered with a
plain JSON response; the server keeps no session and opens no stream, so GET /_mcp and DELETE /_mcp answer
405 Method Not Allowed, and a notification is acknowledged with 202 Accepted.
claude mcp add --transport http locals3 http://localhost:29090/_mcp
# A service with credentials
claude mcp add --transport http locals3 http://localhost:29090/_mcp \
--header "Authorization: Bearer $AWS_SECRET_ACCESS_KEY"
| Tool | Arguments | What it does |
|---|---|---|
list_buckets | The buckets, with their creation date and region. | |
list_objects | bucket, prefix, delimiter (/ by default, "" for none), max_keys (100 by default, up to 1000), continuation_token | A page of the objects under a prefix, and the folders below it as commonPrefixes, like ListObjectsV2. |
read_object | bucket, key, range (e.g. bytes=0-1023, bytes=4096-, bytes=-512), max_bytes (256 KiB by default, up to 4 MiB), version_id | The content of an object: text as text, an image as image content, anything else as a base64 blob. Content whose type says nothing (application/octet-stream, or none, and an extension that doesn't name one) is text only when it is valid UTF-8 without a NUL byte. A read that stops at max_bytes says so and names the range of the rest; text stops before a UTF-8 character it would cut, which starts the rest. The size, ETag and version are those of the content returned. |
put_object | bucket, key, content, encoding (utf-8 or base64), content_type | Store an object, with the content type of its extension unless one is given. |
presign | bucket, key, method (GET or PUT), expires_in (3600 seconds by default, up to 7 days) | A presigned URL of the object, on the host the agent reached the service at, like Share of the console. A service without credentials answers the plain URL, which doesn't expire. |
delete_object | bucket, key, version_id | Delete an object, or put a delete marker in a versioned bucket, like DeleteObject. |
create_bucket | bucket | Create a bucket in the default region, under the naming rules of Amazon S3. |
A tool answers its result as structuredContent and, for the clients that read text alone, as the same JSON in a text
block. A tool that fails, e.g. on a key that doesn't exist, answers isError: true with the S3 error code and message,
e.g. NoSuchKey: ..., so the model reads it and corrects its call.
The objects are also resources: resources/list lists the objects of every bucket, 100 at a time with a cursor,
as s3://<bucket>/<key> with the key percent-encoded, resources/templates/list answers the template
s3://{bucket}/{key}, and resources/read reads up to 4 MiB of one, as text or as a base64 blob. An unknown object is
answered with the error -32002.
The MCP server is guarded like the console, rather than by a signature, which an agent can't make: a service with
credentials wants Authorization: Bearer <secret access key>, or HTTP Basic authentication with the access key ID and
the secret access key, and answers anything else with 401; a service without credentials answers everyone who
reaches the port, as it does unsigned S3 requests. A request that carries an Origin header, which only a browser
sends, is answered only from a page of localhost, 127.0.0.1, [::1] or the host the request addressed, and
403 Forbidden otherwise, so that a page of another site can't reach a service on the loopback address by rebinding
its DNS name. An unsupported MCP-Protocol-Version header is answered with 400.
The tools change the data through the same services as the S3 API, so a write reaches the
change listeners like any other. Its requests are recorded in the
statistics as the operation Mcp. Like /_admin/ui, /_mcp is a path of the service rather than of a bucket
named _mcp while the server is on. S3 Vectors aren't served through MCP yet.
Admin endpoints
A running service answers a few endpoints for local development and tests, with JSON:
| Endpoint | Description |
|---|---|
GET /_admin/stats | The optional features that are on, as features, e.g. ["vector", "s3Tables", "kms", "sts", "console", "website"], and mcp when the MCP server is on, out of those that can be turned off, icebergCatalog and website. The amount of data (buckets, objects, object versions, delete markers, object bytes, multipart uploads in progress, vector buckets, indexes and vectors), how much object metadata the service keeps in heap (loadedObjects and loadedObjectMetadataBytes, see Opening a large data path), the requests in flight, and per operation, e.g. PutObject, the number of requests, the 4xx and 5xx responses, the requests per second of the last minute, and the average, p50, p90, p99 and max latency in milliseconds. The percentiles are estimated from a histogram whose buckets are about 19% wide, interpolated within the bucket, so they are accurate to about 19% and never outside the smallest and the largest latency recorded. notImplemented counts the requests answered 501 NotImplemented, by operation, e.g. SelectObjectContent, or by method, path shape and query parameters, e.g. GET /{bucket}?analytics, for a request without a route. |
GET /_admin/requests?limit=n | The last 100 requests, the most recent first: time, method, URI, operation, status, latency and x-amz-request-id. The values of the credentials of presigned URLs are hidden. |
POST /_admin/lifecycle | Apply the lifecycle configurations of the buckets, which LocalS3 never does by itself, and answer the actions taken. bucket=name applies one bucket's only; now=2030-01-01T00:00:00Z applies them at that instant, or days=31 that many days from now, instead of the current time. See lifecycle configuration. |
GET /_admin/snippets?bucket=&key= | The configuration that DuckDB, s5cmd and the AWS CLI (environment variables, and a profile), boto3, Polars (storage_options), PyIceberg and Spark need to reach the service, as JSON: endpoint, region, icebergCatalog, snippets (each with id, title, language and content), and with bucket the DuckDB query of that bucket, prefix or object as duckdbQuery. The endpoint is the host the request addressed, so a client in a container is given the name it reaches the service by; plain HTTP unless the service serves HTTPS alone. The snippets carry the credentials of the service, which only a caller that signed with them receives, and a placeholder key pair if it has none. |
GET /_admin/snippets/<id> | One snippet as text, for duckdb, env (the exports of s5cmd, the AWS CLI, the AWS SDKs and object_store), aws-cli, boto3, polars, pyiceberg or spark, to paste, or to hand to the client: duckdb -init <(curl -s localhost:29090/_admin/snippets/duckdb) opens a DuckDB that reaches the service. Takes bucket and key too, after which the DuckDB script ends with a query of that object, which curl ... | duckdb runs and exits. |
POST /_admin/reset | Replace the data of an IN_MEMORY service with the data it started with, i.e. none, or the initial data of its data path, and create the AWS_BUCKETS again. The requests recorded for the statistics are forgotten too. A PERSISTENCE service answers 409 Conflict, since a reset would delete its data path. |
A request signed with temporary credentials that carry a
session policy is answered 403 AccessDenied by every admin endpoint, so
that credentials scoped to a table can't reset the service; temporary credentials without a policy are answered like
the key pair of the service.
Resetting a service between the tests that share it is much quicker than restarting it. The requests in progress
are finished first, and the requests that arrive meanwhile wait for the reset. An embedded service is reset with
LocalS3#reset(), and its statistics are read with LocalS3#statistics().
curl -s http://localhost:29090/_admin/stats
curl -s "http://localhost:29090/_admin/snippets/duckdb?bucket=lake&key=events.parquet" | duckdb
duckdb -init <(curl -s http://localhost:29090/_admin/snippets/duckdb)
eval "$(curl -s http://localhost:29090/_admin/snippets/env)" && s5cmd ls
curl -s -X POST http://localhost:29090/_admin/reset
curl -s -X POST "http://localhost:29090/_admin/lifecycle?days=31"
The latency of a request is measured from when its body is received until its response is written. The health check
and the admin endpoints aren't recorded. Unlike the health check, the admin endpoints must be signed if credentials
are configured, e.g. with curl --aws-sigv4 "aws:amz:us-east-1:s3" --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" http://localhost:29090/_admin/stats.
Error responses
A failed S3 request is answered with the usual S3 XML error, e.g. 500 InternalError, and a failed S3 Vectors request
with its JSON error. An Error thrown while a request is handled, e.g. OutOfMemoryError or StackOverflowError, isn't
an Exception and reaches the default handler of the HTTP router instead, which answers 500 with the full stack
trace as text/plain; charset=utf-8. That helps local debugging but exposes the internals of the service, e.g. class
names and paths, to whoever reaches the port, so don't expose LocalS3 to an untrusted network. The other errors of the
router, a request without a route (404) and a failure outside of a handler (500), are plain text as well, so the
request path or exception message they echo is never rendered as HTML.