从node.js调用Sagemaker Tensorflow Resnet50端点 - python

问题

我想知道node.js中与该Python代码等效的内容:

from keras.preprocessing import image
from PIL import Image
from keras.applications.resnet50 import preprocess_input

raw_img = image.load_img("some/path").resize((224, 224), Image.NEAREST)
img = preprocess_input(image.img_to_array(raw_img))

语境

我将Keras的ResNet50模型上传到SageMaker端点。我可以使用以下代码从Python调用它:

import json
import boto3
import numpy as np
import io

client = boto3.client('runtime.sagemaker')

from keras.preprocessing import image
from PIL import Image
from keras.applications.resnet50 import preprocess_input

raw_img = image.load_img("some/path").resize((224, 224), Image.NEAREST)
img = preprocess_input(image.img_to_array(raw_img))

response = client.invoke_endpoint(
  EndpointName="SAGEMAKER_ENDPOINT_NAME",
  Body=json.dumps({ "instances": [ img.tolist() ] }),
  ContentType="application/json"
)

现在,我需要在node.js中执行相同的操作。我想出了如何使用aws-sdk到达终点:

import * as aws from 'aws-sdk';
const sageMaker = new aws.SageMakerRuntime({
  region: 'ap-northeast-1'
});
sageMaker.invokeEndpoint({
  EndpointName: endpointName,
  Body: input,
  ContentType: "application/json",
}, (error, res) => {
  if (error) { return reject(error); }
  // YEAH
})

但是我无法弄清楚如何生成input json,也就是这个python代码段的等效项:

from keras.preprocessing import image
from PIL import Image
from keras.applications.resnet50 import preprocess_input

raw_img = image.load_img("some/path").resize((224, 224), Image.NEAREST)
img = preprocess_input(image.img_to_array(raw_img))

有没有可以实现相同目标的库,还是我需要重新发明轮子?

参考方案

答案是:我不得不重新设计轮子。
这是到达端点的工作需求(对于ResNet50网络而言):

将源图像解码为像素
转换为3个频道(删除
调整为224x224
将像素标准化为[-1,1]
转换为3D数组(SageMaker中的默认格式)

这是我最终得到的代码(使用打字稿):

import * as tf from '@tensorflow/tfjs';

interface INormalizationOptions {
  numberOfChannels: number;
  imageSize: number;
}

interface IImagePixels {
  width: number, height: number, numberOfChannels: number,
  pixels: ndarray
}

export async function preprocessImage(filePath: string, o: INormalizationOptions) {
  const img = await getImagePixels(filePath);

  // Converts the ndarray to tensor, while ignoring extra channels
  const rawTensor = imageToTensor(img, o);

  console.log(`Size of image: ${img.width} ${img.height}`)
  console.log(`Number of color components actually read: ${img.pixels.data.length} (expected: ${img.width*img.height*img.numberOfChannels})`)
  console.log(`Number of pixels: ${img.width*img.height}`);

  // Normalizes the Pixels from [0,255] to [-1,1]
  const normalizedTensor: tf.Tensor3D = rawTensor.toFloat().sub(255/2).div(255/2);

  // Resizes the inage Size to square of IMAGE_SIZE
  const alignCorner = true;
  const resizedInput = tf.image.resizeBilinear(
    normalizedTensor,
    [o.imageSize, o.imageSize],
    alignCorner
  )

  const reshapedTensor: tf.Tensor3D = resizedInput.reshape([ o.imageSize, o.imageSize, o.numberOfChannels ])

  return tensor3dToArray3d(reshapedTensor);
}

这使用了一些实用程序功能。

getImagePixels

async function getImagePixels(filePath: string) {
  const _getPixels = require('get-pixels');
  return new Promise<IImagePixels>( (resolve, reject) => {
    _getPixels( filePath, (error: Error | null, pixels: ndarray) => {
      if (error) { return reject(error); }

      resolve({
        width: pixels.shape[0],
        height: pixels.shape[1],
        numberOfChannels: pixels.shape[2],
        pixels: pixels
      })
    });
  })
}

imageToTensor

import ndarray from 'ndarray';

function imageToTensor(img: IImagePixels, o: INormalizationOptions) {
  const rawTensorValues = new Int32Array(img.width * img.height * o.numberOfChannels);
  const rawTensor = tf.tensor3d(rawTensorValues, [ img.width, img.height, o.numberOfChannels ], 'int32');

  // This only uses the first CHANNEL_ND
  for (let row=0; row<img.height; row++) {
    for (let col=0; col<img.width; col++){
      for (let channel =0; channel<o.numberOfChannels; channel++) {
        const pixel = img.pixels.get(row,col,channel);
        if (!isNumeric(pixel)) { throw new Error(`Bad pixel: ${pixel}`) }
        const offset = row * img.width * img.numberOfChannels + col * img.numberOfChannels + channel
        rawTensorValues[offset] = pixel;
      }
    }
  }

  return rawTensor;
}

tensor3dToArray3d

async function tensor3dToArray3d(t: tf.Tensor3D) {
  const dataAs3dArray = new Array<number[][]>();
  const data = await t.data();

  const expectedSizeOfArray = t.shape[0] * t.shape[1] * t.shape[2];


  console.log(`Got vector of size ${t.shape[0]} ${t.shape[1]} ${t.shape[2]}, that is, ${expectedSizeOfArray} data`);
  console.log(`Actual size of data: ${data.length}`);

  for (let i=0; i<t.shape[0]; i++) {
    dataAs3dArray[i] = new Array<number[]>();
    for (let j=0; j<t.shape[1]; j++) {
      dataAs3dArray[i][j] = new Array<number>();
      for (let k=0; k<t.shape[2]; k++ ) {
        const ijk =
            i * t.shape[0] * t.shape[1] * t.shape[2]
          + j * t.shape[1] * t.shape[2]
          + k;

        const datum = Math.random(); // Number( data[ijk] );
        if (!isNumeric(datum)) { throw new Error(`Invalid datum at ${i},${j},${k}: ${datum}`) }
        dataAs3dArray[i][j][k] = datum;
      }
    }
  }
  return dataAs3dArray;
}

function isNumeric(n: any) {
  return !isNaN(parseFloat(n)) && isFinite(n);
}

有没有一种方法可以有效地矢量化图像上的Tensorflow操作? - python

Tensorflow有大量的变换,可以应用于表示图像([高度,宽度,深度])(例如tf.image.rot90()或tf.image.random_flip_left_right())的3D张量。我知道它们应与队列一起使用,因此它们只能在一个图像上运行。但是,是否有一种方法可以对操作进行矢量化处理,以将4D张量([batch_size,height,widt…

TensorFlow操作,在官方API中找不到 - python

最近,我尝试重复学习Nvidia在GitHub上发布的代码-progressive_growing_of_gans。但是,我发现以下几种基于官方API找不到的操作参考。feed_dict = {} setter = tf.assign(var, tf.placeholder(var.dtype, var.shape, 'new_value'…

Tensorflow安装成功但不起作用 - python

[libprotobuf致命  google / protobuf / src / google / protobuf / stubs / common.cc:67]此程序  需要协议缓冲区运行时库的3.3.0版本,但是  安装的版本是3.0.0。请更新您的图书馆。如果你  自己编译程序,确保头文件来自  与链接时库相同版本的协议缓冲区。  (“ googl…

安装Tensor Flow泊坞映像 - python

我正在使用下面的链接安装张量流docker映像,对于最后一步,当我运行sudo docker pull tensorflow/tensorflow:latest-gpu-jupyter时,出现以下错误消息:Error response from daemon: manifest for tensorflow/tensorflow:devel-gpu-jupy…

无法在TensorFlow中转换部分转换的张量 - python

TensorFlow中有很多方法需要指定形状,例如truncated_normal:tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None) 我有一个用于输入形状[None,784]的占位符,其中第一维为“无”,因为批量大小可以变化。我可以…