View Source Dynamo.Stream (Dynamo v0.2.0)

Lazy streams over paginated Query/Scan results.

# all orders of a user, lazily page by page
MyApp.Dynamo.stream(%Order{user_id: "u1"})
|> Stream.filter(&(&1.status == "open"))
|> Enum.take(100)

# whole-table scan
MyApp.Dynamo.stream(User, filter: "attribute_exists(#e)", names: %{"#e" => "email"})
|> Enum.each(&process/1)

# truly parallel scan: N segments fetched concurrently, results interleaved
MyApp.Dynamo.stream(User, segments: 8, max_concurrency: 4)
|> Enum.count()

Pass a struct to query (partition key from the struct, see Dynamo.Request.query/2) or a schema module to scan.

Options: all Dynamo.Request query/scan options, plus

  • :page_size – maps to DynamoDB Limit per request (default: unset)
  • :segments – scan only; number of parallel segments (default 1)
  • :max_concurrency – scan only; concurrent segment fetches (default segments)

Errors raise %Dynamo.Error{} while the stream is consumed (the only way to signal failure from inside a stream). Scans consume read capacity for every item examined – stream lazily, but do not stream casually.

Summary

Functions

new(repo, struct, opts)

@spec new(module(), struct() | module(), keyword()) :: Enumerable.t()