Cloudflare Developers•7mo ago

D1PreparedStatement API

the conversion from D1Type -> JsValue is done once per item per row, which means that every time you call bind_refs, all the D1Types have to be converted. Say we have 4 things we want to bind, and 2 are static, this means that every time we are passing all 4 to js and back. If we're clever with our iterator and are able to use jsvalue, we can instead always return the same 2 references to the static jsvalues and and only convert the 2 dynamic ones every bind_refs

28 Replies

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

yeah, that would work too I like this approach regardless because you're able to make various changes in the future without the API changing the D1Type approach I mean

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

yeah for sure, it was a bit of an adventure for me trying to figure out how to manage a blob when I started looking at this stuff, so being able to have rust's type system to help here would be awesome actually one thing Number(f64) I believe currently that is accurate because of the way this is managed JS side, but this could change in the future? Idk what your guys' plans are but sqlite does technically support i64, and I assume at some point the bindings will be tweaked to allow that So maybe take inspiration from sqlite and go with Real(f64) and then either later add Integer(i64) or add it now and panic when it's used or something?

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

I'm thinking that I liked the way the enum used rusts type system in case someone wanted to bypass JS entirely at some point, and I'm trying to think if there's a nice way to still make that work together with this approach I also just generally don't really like how we lose the values after convert to jsvalue I kinda like something like this

pub enum D1Type {
    Null,
    Real(f64),
    Integer(i32),
    Text(String),
    Boolean(bool),
    Blob(Vec<u8>),
}


pub struct D1TypeObj {
    pub value : D1Type,
    js_value : JsValue
}

impl D1TypeObj {
    pub fn new(value : D1Type) -> Self {
        Self {
            value,
            js_value : match value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(&s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(&a).unwrap()
            }
        }
    }
}

impl From<&D1TypeObj> for &JsValue {
    fn from(d1_type_obj: &D1TypeObj) -> Self {
        &d1_type_obj.js_value
    }
}

impl From<&D1TypeObj> for &D1Type {
    fn from(d1_type_obj: &D1TypeObj) -> Self {
        &d1_type_obj.value
    }
}

impl From<D1Type> for D1TypeObj {
    fn from(value: D1Type) -> Self {
        Self::new(value)
    }
}

pub enum D1Type {
    Null,
    Real(f64),
    Integer(i32),
    Text(String),
    Boolean(bool),
    Blob(Vec<u8>),
}


pub struct D1TypeObj {
    pub value : D1Type,
    js_value : JsValue
}

impl D1TypeObj {
    pub fn new(value : D1Type) -> Self {
        Self {
            value,
            js_value : match value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(&s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(&a).unwrap()
            }
        }
    }
}

impl From<&D1TypeObj> for &JsValue {
    fn from(d1_type_obj: &D1TypeObj) -> Self {
        &d1_type_obj.js_value
    }
}

impl From<&D1TypeObj> for &D1Type {
    fn from(d1_type_obj: &D1TypeObj) -> Self {
        &d1_type_obj.value
    }
}

impl From<D1Type> for D1TypeObj {
    fn from(value: D1Type) -> Self {
        Self::new(value)
    }
}

except for the name I'm terrible at naming things in case that isn't abundantly obvious But this approach means that if someone doesn't care, they can just use D1Type If they want some 'caching' they use D1TypeObj

Isaac McFadyen•7mo ago

JS supports i64 with BigInts but I'm uncertain if they can be used in D1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt (BigInts are arbitrary precision)

Enduriel•7mo ago

iirc they weren't, I loooked into this at one point

Enduriel•7mo ago

Enduriel•7mo ago

tbh the & impls aren't necessary actually the obj doesn't even need ownership of the value in fact it shouldn't have it

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

ye sorry I'm working on making it all work well rn

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

you can mark the enum as [non_exhaustive] but this is useful for debugging imo and in principle this information doesn't really make sense to be lost imo

pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(Vec<u8>),
}

pub struct D1TypeObj<'a> {
    pub value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(*f),
                D1Type::Integer(i) => JsValue::from_f64(*i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(*b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(Vec<u8>),
}

pub struct D1TypeObj<'a> {
    pub value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(*f),
                D1Type::Integer(i) => JsValue::from_f64(*i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(*b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

pub fn bind_refs<'a, T, U: 'a>(&self, values: T) -> Result<Self>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: Into<D1TypeObj<'a>>,
{
    let array: Array = values.into_iter().map(|&t| {Into::<D1TypeObj>::into(t).js_value}).collect::<Array>();

    match self.0.bind(array) {
        Ok(stmt) => Ok(D1PreparedStatement(stmt)),
        Err(err) => Err(Error::from(err)),
    }
}

pub fn bind_refs<'a, T, U: 'a>(&self, values: T) -> Result<Self>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: Into<D1TypeObj<'a>>,
{
    let array: Array = values.into_iter().map(|&t| {Into::<D1TypeObj>::into(t).js_value}).collect::<Array>();

    match self.0.bind(array) {
        Ok(stmt) => Ok(D1PreparedStatement(stmt)),
        Err(err) => Err(Error::from(err)),
    }
}

is what I have right now, which works with both

    let stmt = unbound_stmt.bind_refs(&[&D1Type::Text("Ryan Upton")])?;
    let D1TypeObj = D1Type::Text("Ryan Upton");
    let stmt = unbound_stmt.bind_refs(&[&D1TypeObj])?; // Bind by value

    let stmt = unbound_stmt.bind_refs(&[&D1Type::Text("Ryan Upton")])?;
    let D1TypeObj = D1Type::Text("Ryan Upton");
    let stmt = unbound_stmt.bind_refs(&[&D1TypeObj])?; // Bind by value

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

yes, true Tbh, maybe I'm overthinking it, but I'm trying to make this both inspectable and extendable in the future. maybe it's okay if it's not inspectable, I also just really like working with rust enums as doing stuff like this just feels great

js_value : match value {
    D1Type::Null => JsValue::null(),
    D1Type::Real(f) => JsValue::from_f64(*f),
    D1Type::Integer(i) => JsValue::from_f64(*i as f64),
    D1Type::Text(s) => JsValue::from_str(s),
    D1Type::Boolean(b) => JsValue::from_bool(*b),
    D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
}

js_value : match value {
    D1Type::Null => JsValue::null(),
    D1Type::Real(f) => JsValue::from_f64(*f),
    D1Type::Integer(i) => JsValue::from_f64(*i as f64),
    D1Type::Text(s) => JsValue::from_str(s),
    D1Type::Boolean(b) => JsValue::from_bool(*b),
    D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
}

pub fn batch_bind<'a, T: 'a, U: 'a, V: 'a>(&self, values: T) -> Result<Vec<Self>>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: IntoIterator<Item = &'a &'a V>,
    &'a V: Into<D1TypeObj<'a>>,
{
    values
        .into_iter()
        .map(|&batch| self.bind_refs(batch))
        .collect()
}

pub fn batch_bind<'a, T: 'a, U: 'a, V: 'a>(&self, values: T) -> Result<Vec<Self>>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: IntoIterator<Item = &'a &'a V>,
    &'a V: Into<D1TypeObj<'a>>,
{
    values
        .into_iter()
        .map(|&batch| self.bind_refs(batch))
        .collect()
}

I think copilot wrote that part for me which is why both of those were bad lol final proposal

pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(&'a [u8]),
}

pub struct D1TypeObj<'a> {
    pub value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match *value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(&'a [u8]),
}

pub struct D1TypeObj<'a> {
    pub value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match *value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

pub fn bind_refs<'a, T, U: 'a>(&self, values: T) -> Result<Self>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: Into<D1TypeObj<'a>>,
{
    let array: Array = values.into_iter().map(|&t| {Into::<D1TypeObj>::into(t).js_value}).collect::<Array>();

    match self.0.bind(array) {
        Ok(stmt) => Ok(D1PreparedStatement(stmt)),
        Err(err) => Err(Error::from(err)),
    }
}

pub fn batch_bind<'a, T: 'a, U: 'a, V: 'a>(&self, values: T) -> Result<Vec<Self>>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: IntoIterator<Item = &'a &'a V>,
    &'a V: Into<D1TypeObj<'a>>,
{
    values
        .into_iter()
        .map(|&batch| self.bind_refs(batch))
        .collect()
}

pub fn bind_refs<'a, T, U: 'a>(&self, values: T) -> Result<Self>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: Into<D1TypeObj<'a>>,
{
    let array: Array = values.into_iter().map(|&t| {Into::<D1TypeObj>::into(t).js_value}).collect::<Array>();

    match self.0.bind(array) {
        Ok(stmt) => Ok(D1PreparedStatement(stmt)),
        Err(err) => Err(Error::from(err)),
    }
}

pub fn batch_bind<'a, T: 'a, U: 'a, V: 'a>(&self, values: T) -> Result<Vec<Self>>
where
    T: IntoIterator<Item = &'a &'a U>,
    &'a U: IntoIterator<Item = &'a &'a V>,
    &'a V: Into<D1TypeObj<'a>>,
{
    values
        .into_iter()
        .map(|&batch| self.bind_refs(batch))
        .collect()
}

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

I still hate the name D1TypeObj, but sounds good could also be really naughty and implement into_iter for &D1Type, returning an Iterator<&D1Type> lol would mean no wrappers would be needed for a single &D1Type bind_refs

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

Right, I forgot Rust allows field mutation directly. Two solutions, either implement From<&D1TypeObj> for &D1Type or add a getter. I think I prefer the trait personally, makes it a little more ergonomic because in my mind they are still ~ the same thing, the D1TypeObj is just a precalculated version of the D1Type maybe a good name would be going in the direction of calculated or cached

#[non_exhaustive]
pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(&'a [u8]),
}

pub struct D1TypeObj<'a> {
    value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match *value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

impl<'a> From<&'a D1TypeObj<'a>> for &D1Type<'a> {
    fn from(obj: &'a D1TypeObj<'a>) -> Self {
        obj.value
    }
}

#[non_exhaustive]
pub enum D1Type<'a> {
    Null,
    Real(f64),
    Integer(i32),
    Text(&'a str),
    Boolean(bool),
    Blob(&'a [u8]),
}

pub struct D1TypeObj<'a> {
    value : &'a D1Type<'a>,
    js_value : JsValue
}

impl<'a> D1TypeObj<'a> {
    pub fn new(value : &'a D1Type) -> D1TypeObj<'a> {
        Self {
            value,
            js_value : match *value {
                D1Type::Null => JsValue::null(),
                D1Type::Real(f) => JsValue::from_f64(f),
                D1Type::Integer(i) => JsValue::from_f64(i as f64),
                D1Type::Text(s) => JsValue::from_str(s),
                D1Type::Boolean(b) => JsValue::from_bool(b),
                D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap()
            }
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for D1TypeObj<'a> {
    fn from(value: &'a D1Type<'a>) -> Self {
        Self::new(value)
    }
}

impl<'a> From<&'a D1TypeObj<'a>> for &D1Type<'a> {
    fn from(obj: &'a D1TypeObj<'a>) -> Self {
        obj.value
    }
}

Maybe D1TypeCached, I'd say that's a good descriptor for what it actually does

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

yup, looks amazing now, only 2 tiny suggestions at this point are to rename PreparedArgument to D1PreparedArgument since everything else around there is D1XYZ and then a very pedantic thing but I think

impl<'a> From<&'a D1Type<'a>> for JsValue {
    fn from(value: &'a D1Type<'a>) -> Self {
        match *value {
            D1Type::Null => JsValue::null(),
            D1Type::Real(f) => JsValue::from_f64(f),
            D1Type::Integer(i) => JsValue::from_f64(i as f64),
            D1Type::Text(s) => JsValue::from_str(s),
            D1Type::Boolean(b) => JsValue::from_bool(b),
            D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap(),
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for JsValue {
    fn from(value: &'a D1Type<'a>) -> Self {
        match *value {
            D1Type::Null => JsValue::null(),
            D1Type::Real(f) => JsValue::from_f64(f),
            D1Type::Integer(i) => JsValue::from_f64(i as f64),
            D1Type::Text(s) => JsValue::from_str(s),
            D1Type::Boolean(b) => JsValue::from_bool(b),
            D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap(),
        }
    }
}

is nicer than

impl<'a> From<&'a D1Type<'a>> for JsValue {
    fn from(value: &'a D1Type<'a>) -> Self {
        match value {
            D1Type::Null => JsValue::null(),
            D1Type::Real(f) => JsValue::from_f64(*f),
            D1Type::Integer(i) => JsValue::from_f64(*i as f64),
            D1Type::Text(s) => JsValue::from_str(s),
            D1Type::Boolean(b) => JsValue::from_bool(*b),
            D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap(),
        }
    }
}

impl<'a> From<&'a D1Type<'a>> for JsValue {
    fn from(value: &'a D1Type<'a>) -> Self {
        match value {
            D1Type::Null => JsValue::null(),
            D1Type::Real(f) => JsValue::from_f64(*f),
            D1Type::Integer(i) => JsValue::from_f64(*i as f64),
            D1Type::Text(s) => JsValue::from_str(s),
            D1Type::Boolean(b) => JsValue::from_bool(*b),
            D1Type::Blob(a) => serde_wasm_bindgen::to_value(a).unwrap(),
        }
    }
}

especially since this way you don't pass &&str and &&[u8] Big fan of how this turned out also D1PreparedArgument seems so obvious in hindsight considering all the other types around there it's really a painful reminder of how bad I am at naming things 😅 Actually thoughts on this kinda thing to really maximize ergonomics?

impl<'a> Into<D1Type<'a>> for &'a str {
    fn into(self) -> D1Type<'a> {
        D1Type::Text(self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a f64 {
    fn into(self) -> D1Type<'a> {
        D1Type::Real(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a i32 {
    fn into(self) -> D1Type<'a> {
        D1Type::Integer(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a bool {
    fn into(self) -> D1Type<'a> {
        D1Type::Boolean(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a [u8] {
    fn into(self) -> D1Type<'a> {
        D1Type::Blob(self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a str {
    fn into(self) -> D1Type<'a> {
        D1Type::Text(self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a f64 {
    fn into(self) -> D1Type<'a> {
        D1Type::Real(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a i32 {
    fn into(self) -> D1Type<'a> {
        D1Type::Integer(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a bool {
    fn into(self) -> D1Type<'a> {
        D1Type::Boolean(*self)
    }
}

impl<'a> Into<D1Type<'a>> for &'a [u8] {
    fn into(self) -> D1Type<'a> {
        D1Type::Blob(self)
    }
}

actually the compiler doesn't seem to be clever enough for that That's really unfortunate, maybe there's a clever way to do that in rust but I don't have that much experience with rust traits I mean tbf something like this is useful even if the compiler currently can't automatically figure out the correct types imo

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

True man it's so much nicer to have

.bind_refs(&D1Type::Text(&lang.language_code))?;

.bind_refs(&D1Type::Text(&lang.language_code))?;

instead of

.bind(&[serde_wasm_bindgen::to_value(&lang.language_code)?])?

.bind(&[serde_wasm_bindgen::to_value(&lang.language_code)?])?

I am having some trouble with batch_bind ok, the simple example works, but trying to be clever with the iterator seems to be difficult... Am I incompetent? I can't get this to work

let query = db.prepare("SELECT * FROM people WHERE name = ?");
let texts = vec!["a", "b"];
let test = texts.iter().map(|text| D1Type::Text(text).into_iter());
let queries = query.batch_bind(test.into())?;

let query = db.prepare("SELECT * FROM people WHERE name = ?");
let texts = vec!["a", "b"];
let test = texts.iter().map(|text| D1Type::Text(text).into_iter());
let queries = query.batch_bind(test.into())?;

this works

let queries = untranslated_texts
    .iter()
    .map(|text| query.bind_refs(&D1Type::Text(text)))
    .collect::<Result<Vec<D1PreparedStatement>>>()?;

let queries = untranslated_texts
    .iter()
    .map(|text| query.bind_refs(&D1Type::Text(text)))
    .collect::<Result<Vec<D1PreparedStatement>>>()?;

imo, if it's not possible to make the batch_bind work nicely with the previous example, I'm not sure it's necessary at all, this is fine imo

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

batch_bind is asking for type annotations, which makes it pretty much useless at that point imo this isn't the issue though

Unknown User•7mo ago

Message Not Public

Enduriel•7mo ago

this is kinda neat I'd argue this is worse than doing it with bind_refs directly, but maybe that's just me I don't think we can get any better with rust's inference though unfortunately

Gaming

Programming

D1PreparedStatement API