Modular•10mo ago

Different outputs with random numbers

Hi, I am running code to find the covariance of a matrix from scratch. When the code is run I sometimes get large or small random unexpected numbers or even infinity. It seems to change every time the code is run even when there is no random element in the code, and should produce the same output. Many thanks for any advice. Code snippet:

fn CovarienceMatrix(borrowed self) -> Matrix:
        let transposed_matrix = self.Transpose()
        var avg_dataset = Matrix(transposed_matrix.height, 1, Float32(0))
        var varience = Matrix(transposed_matrix.height, 1, Float32(0))
        var covarince_matrix = Matrix(transposed_matrix.height, transposed_matrix.height, Float32(0))
        for i in range(transposed_matrix.height):
            @parameter
            fn sum_total[num_elmts: Int](j: Int):
                avg_dataset[i,0] = avg_dataset[i,0] + transposed_matrix.load[num_elmts](i,j).reduce_add[1]()
            vectorize[simd_width, sum_total](transposed_matrix.width)
            avg_dataset[i,0] = avg_dataset[i,0] / transposed_matrix.width
        for i in range(transposed_matrix.height):
            @parameter
            fn normalise_total[num_elmts: Int](j: Int):
                varience[i,0] = varience[i,0] + pow(transposed_matrix.load[num_elmts](i,j) - avg_dataset[i,0], 2).reduce_add[1]()
            vectorize[simd_width, normalise_total](transposed_matrix.width)
            varience[i,0] = sqrt(varience[i,0] / transposed_matrix.width)
        for i in range(transposed_matrix.height):
            for j in range(transposed_matrix.height):
                covarince_matrix[i, j] = varience[i,0] * varience[j,0]
        return covarince_matrix

fn CovarienceMatrix(borrowed self) -> Matrix:
        let transposed_matrix = self.Transpose()
        var avg_dataset = Matrix(transposed_matrix.height, 1, Float32(0))
        var varience = Matrix(transposed_matrix.height, 1, Float32(0))
        var covarince_matrix = Matrix(transposed_matrix.height, transposed_matrix.height, Float32(0))
        for i in range(transposed_matrix.height):
            @parameter
            fn sum_total[num_elmts: Int](j: Int):
                avg_dataset[i,0] = avg_dataset[i,0] + transposed_matrix.load[num_elmts](i,j).reduce_add[1]()
            vectorize[simd_width, sum_total](transposed_matrix.width)
            avg_dataset[i,0] = avg_dataset[i,0] / transposed_matrix.width
        for i in range(transposed_matrix.height):
            @parameter
            fn normalise_total[num_elmts: Int](j: Int):
                varience[i,0] = varience[i,0] + pow(transposed_matrix.load[num_elmts](i,j) - avg_dataset[i,0], 2).reduce_add[1]()
            vectorize[simd_width, normalise_total](transposed_matrix.width)
            varience[i,0] = sqrt(varience[i,0] / transposed_matrix.width)
        for i in range(transposed_matrix.height):
            for j in range(transposed_matrix.height):
                covarince_matrix[i, j] = varience[i,0] * varience[j,0]
        return covarince_matrix

8 Replies

sora•10mo ago

Could you provide a minimal reproducible example that people can run (include all the code, etc.)?

HarambeOP•10mo ago

This is a reduced code sample that reproduces the problem:

message.txt

Michael K•10mo ago

This code prints matrix of 3s and then matrix of 9s. No unexpected behaviour on M3 Macbook.
You are using simdwidthof to get a SIMD width specific to your system (mine would be 4) so that may be one reason it doesn't reproduce. Adding bounds checks to your load and store methods would confirm whether you are staying in the allocated pointers.

sora•10mo ago

I can reproduce on my machine (Intel Mac with Docker). Trying to reduce the example further now. Reduced:

from algorithm import vectorize

struct Vec:
    var length: Int
    var data: DTypePointer[DType.float32]

    fn __init__(inout self, length: Int):
        self.length = length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        for item in range(self.length):
            self.data.store(item, 1)

    fn __copyinit__(inout self, other: Self):
        self.length = other.length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        memcpy(self.data, other.data, self.length)

    fn __del__(owned self):
        self.data.free()

    fn non_deterministic(self) -> Vec:
        let a = Vec(self.length)
        var b = Vec(self.length)
        for i in range(b.length):
            @parameter
            fn f[simd_width: Int](j: Int):
                b.data.store(i, a.data.load(i))
            vectorize[1, f](self.length)
        return b

fn print_vec(self: Vec):
    for i in range(self.length):
        print_no_newline(self.data[i])
        print_no_newline(",")
    print("\n")

fn main():
    let a = Vec(2)
    let b = a.non_deterministic()
    print_vec(b)

from algorithm import vectorize

struct Vec:
    var length: Int
    var data: DTypePointer[DType.float32]

    fn __init__(inout self, length: Int):
        self.length = length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        for item in range(self.length):
            self.data.store(item, 1)

    fn __copyinit__(inout self, other: Self):
        self.length = other.length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        memcpy(self.data, other.data, self.length)

    fn __del__(owned self):
        self.data.free()

    fn non_deterministic(self) -> Vec:
        let a = Vec(self.length)
        var b = Vec(self.length)
        for i in range(b.length):
            @parameter
            fn f[simd_width: Int](j: Int):
                b.data.store(i, a.data.load(i))
            vectorize[1, f](self.length)
        return b

fn print_vec(self: Vec):
    for i in range(self.length):
        print_no_newline(self.data[i])
        print_no_newline(",")
    print("\n")

fn main():
    let a = Vec(2)
    let b = a.non_deterministic()
    print_vec(b)

Michael K•10mo ago

For the prospective bug report, the smaller reproducer also shows no bad behaviour on M3 Macbook.

sora•10mo ago

Reported here

GitHub

[BUG]: Code showing non deterministic behaviour · Issue #1755 · mo...

Bug description As title. Doesn't seem to reproduce on M3 Mac. Steps to reproduce from algorithm import vectorize struct Vec: var length: Int var data: DTypePointer[DType.float32] fn init(i...

sora•9mo ago

Seems like a simple problem caused by early destruction, wonder how I missed that. Fix: add _ = a before return b. I think you can fix your CovarienceMatrix similarly.

HarambeOP•9mo ago

Updated: The intended fix has solved the problem, thanks for all of the help

Gaming

Programming

Different outputs with random numbers