Cloudflare Developers•5mo ago

Enterprise customer - cannot perform I/O error

Hi all - I was told to come hear from enterprise support as they were not sure what the issue was. We're getting a couple of errors on a worker of ours that interacts with a durable object: 1. Cannot perform I/O on behalf of a different request. I/O objects (such as streams, request/response bodies, and others) created in the context of one request handler cannot be accessed from a different request's handler. This is a limitation of Cloudflare Workers which allows us to improve overall performance. (I/O type: OutgoingFactory) 2. this.viewer.fetch is not a function (this.viewer is the durable object) Can anyone help us diagnose the issue? Unfortunately support weren't able to tell us where or why either of the errors where coming from. The issue isn't constant, only affecting a small portion of our requests. However, because this worker gets so many requests, we're getting a decent amount of errors. Thanks

8 Replies

Hard@Work•5mo ago

This sounds like you are sharing promises in the global scope. I.e.

let someThing;
export default {
  async fetch() {
    someThing = await fetch("https://1.1.1.1/");
    // ...
  }
}

let someThing;
export default {
  async fetch() {
    someThing = await fetch("https://1.1.1.1/");
    // ...
  }
}

CemOP•4mo ago

Hi @Hard@Work | R2 - thanks for the response. However, we're not declaring any variables at the global scope. To help narrow it down, would you be able to tell me if that error comes from interacting with the durable object or not? Or what I/O operation might be throwing the error?

Hard@Work•4mo ago

Can you share a code snippet from where this is happening?

CemOP•4mo ago

@Hard@Work | R2 I'm not sure if this is where the error is coming from but the stack trace seems to suggest this method from our class:

async listViews(): Promise<ViewData[]> {
    let views: ViewData[] = [];
    try {
      const viewsRaw = await this.viewer.fetch(
        constructUrl(`${this.info.id}/list`),
      );
      views = await viewsRaw.json();
    } catch (e: any) {
      this.logger.error('Failed to fetch view list', e);
      throw e;
    }
    return views;
  }

async listViews(): Promise<ViewData[]> {
    let views: ViewData[] = [];
    try {
      const viewsRaw = await this.viewer.fetch(
        constructUrl(`${this.info.id}/list`),
      );
      views = await viewsRaw.json();
    } catch (e: any) {
      this.logger.error('Failed to fetch view list', e);
      throw e;
    }
    return views;
  }

Hard@Work•4mo ago

How do you construct instances of this class? Where are they stored?

CemOP•4mo ago

Hi @Hard@Work | R2 , it is constructed during the Hono router construction under the init() method as such:

export async function constructRouter(
  request: Request,
  env: Bindings,
  logger: Logger,
  ctx: ExecutionContext,
): Promise<Hono<{ Bindings: Bindings }>> {
  const app = new Hono<{ Bindings: Bindings }>();

  // Register logging middleware
  app.use('*', async (event, next) => {
    try {
      logger.send(ctx);
      await next();
    } catch (error: any) {
      logger.error(error);
      logger.send(ctx);
      return event.json(
        {
          message: 'Server error',
        },
        500,
      );
    }
  });
  // Register CORS middleware
  app.use('*', async (event, next) => {
    await next();
    event.res.headers.set(
      'Access-Control-Allow-Origin',
      event.req.header('origin') ?? '*',
    );
    event.res.headers.set('Access-Control-Max-Age', '86400');
    event.res.headers.set(
      'Access-Control-Allow-Methods',
      'GET, PATCH, POST, PUT, DELETE, OPTIONS, HEAD',
    );
    event.res.headers.set(
      'Access-Control-Allow-Headers',
      'origin, content-type, accept, authorization, x-requested-with, x-datadog-trace-id, x-datadog-parent-id, x-datadog-origin, x-datadog-sampling-priority, x-datadog-sampled',
    );
  });

  // Register OPTIONS handler
  app.options('*', (event) => event.text(''));

  // Register User class init middleware
  app.use('*', async (event, next) => {
    try {
      event.env.User = new User(request, env, logger) as User;
      await event.env.User.init();
      await next();
    } catch (err) {
      logger.error(err, {
        user: event.env.User.info.id,
      });
      return event.json(
        {
          errors: {
            status: 401,
            title: 'You must be authenticated to access this resource',
          },
        },
        401,
      );
    }
  });

export async function constructRouter(
  request: Request,
  env: Bindings,
  logger: Logger,
  ctx: ExecutionContext,
): Promise<Hono<{ Bindings: Bindings }>> {
  const app = new Hono<{ Bindings: Bindings }>();

  // Register logging middleware
  app.use('*', async (event, next) => {
    try {
      logger.send(ctx);
      await next();
    } catch (error: any) {
      logger.error(error);
      logger.send(ctx);
      return event.json(
        {
          message: 'Server error',
        },
        500,
      );
    }
  });
  // Register CORS middleware
  app.use('*', async (event, next) => {
    await next();
    event.res.headers.set(
      'Access-Control-Allow-Origin',
      event.req.header('origin') ?? '*',
    );
    event.res.headers.set('Access-Control-Max-Age', '86400');
    event.res.headers.set(
      'Access-Control-Allow-Methods',
      'GET, PATCH, POST, PUT, DELETE, OPTIONS, HEAD',
    );
    event.res.headers.set(
      'Access-Control-Allow-Headers',
      'origin, content-type, accept, authorization, x-requested-with, x-datadog-trace-id, x-datadog-parent-id, x-datadog-origin, x-datadog-sampling-priority, x-datadog-sampled',
    );
  });

  // Register OPTIONS handler
  app.options('*', (event) => event.text(''));

  // Register User class init middleware
  app.use('*', async (event, next) => {
    try {
      event.env.User = new User(request, env, logger) as User;
      await event.env.User.init();
      await next();
    } catch (err) {
      logger.error(err, {
        user: event.env.User.info.id,
      });
      return event.json(
        {
          errors: {
            status: 401,
            title: 'You must be authenticated to access this resource',
          },
        },
        401,
      );
    }
  });

And here is the index.ts

import { constructRouter } from './router';

export { Views } from './views';

export default {
  async fetch(
    request: Request,
    env: Bindings,
    ctx: ExecutionContext,
  ): Promise<Response> {
    // Setup logging to datadog
    const logger = new Logger(
      new DatadogLoggerProvider({
        apiKey: env.DD_API_KEY,
        applicationKey: env.DD_APP_KEY,
        environment: env.ENVIRONMENT,
      }),
    );

    // Construct router
    const app = await constructRouter(request, env, logger, ctx);

    return app.fetch(request, env, ctx);
  },
};

import { constructRouter } from './router';

export { Views } from './views';

export default {
  async fetch(
    request: Request,
    env: Bindings,
    ctx: ExecutionContext,
  ): Promise<Response> {
    // Setup logging to datadog
    const logger = new Logger(
      new DatadogLoggerProvider({
        apiKey: env.DD_API_KEY,
        applicationKey: env.DD_APP_KEY,
        environment: env.ENVIRONMENT,
      }),
    );

    // Construct router
    const app = await constructRouter(request, env, logger, ctx);

    return app.fetch(request, env, ctx);
  },
};

@Hard@Work | R2 ?

Hard@Work•4mo ago

Sorry, I'm not sure what else it could be...

CemOP•4mo ago

well at least we're not missing something obvious! how can we progress from here?

Gaming

Programming

Enterprise customer - cannot perform I/O error