M
Modular•11mo ago
Harambe

Different outputs with random numbers

Hi, I am running code to find the covariance of a matrix from scratch. When the code is run I sometimes get large or small random unexpected numbers or even infinity. It seems to change every time the code is run even when there is no random element in the code, and should produce the same output. Many thanks for any advice. Code snippet: 
fn CovarienceMatrix(borrowed self) -> Matrix:
        let transposed_matrix = self.Transpose()
        var avg_dataset = Matrix(transposed_matrix.height, 1, Float32(0))
        var varience = Matrix(transposed_matrix.height, 1, Float32(0))
        var covarince_matrix = Matrix(transposed_matrix.height, transposed_matrix.height, Float32(0))
        for i in range(transposed_matrix.height):
            @parameter
            fn sum_total[num_elmts: Int](j: Int):
                avg_dataset[i,0] = avg_dataset[i,0] + transposed_matrix.load[num_elmts](i,j).reduce_add[1]()
            vectorize[simd_width, sum_total](transposed_matrix.width)
            avg_dataset[i,0] = avg_dataset[i,0] / transposed_matrix.width
        for i in range(transposed_matrix.height):
            @parameter
            fn normalise_total[num_elmts: Int](j: Int):
                varience[i,0] = varience[i,0] + pow(transposed_matrix.load[num_elmts](i,j) - avg_dataset[i,0], 2).reduce_add[1]()
            vectorize[simd_width, normalise_total](transposed_matrix.width)
            varience[i,0] = sqrt(varience[i,0] / transposed_matrix.width)
        for i in range(transposed_matrix.height):
            for j in range(transposed_matrix.height):
                covarince_matrix[i, j] = varience[i,0] * varience[j,0]
        return covarince_matrix
fn CovarienceMatrix(borrowed self) -> Matrix:
        let transposed_matrix = self.Transpose()
        var avg_dataset = Matrix(transposed_matrix.height, 1, Float32(0))
        var varience = Matrix(transposed_matrix.height, 1, Float32(0))
        var covarince_matrix = Matrix(transposed_matrix.height, transposed_matrix.height, Float32(0))
        for i in range(transposed_matrix.height):
            @parameter
            fn sum_total[num_elmts: Int](j: Int):
                avg_dataset[i,0] = avg_dataset[i,0] + transposed_matrix.load[num_elmts](i,j).reduce_add[1]()
            vectorize[simd_width, sum_total](transposed_matrix.width)
            avg_dataset[i,0] = avg_dataset[i,0] / transposed_matrix.width
        for i in range(transposed_matrix.height):
            @parameter
            fn normalise_total[num_elmts: Int](j: Int):
                varience[i,0] = varience[i,0] + pow(transposed_matrix.load[num_elmts](i,j) - avg_dataset[i,0], 2).reduce_add[1]()
            vectorize[simd_width, normalise_total](transposed_matrix.width)
            varience[i,0] = sqrt(varience[i,0] / transposed_matrix.width)
        for i in range(transposed_matrix.height):
            for j in range(transposed_matrix.height):
                covarince_matrix[i, j] = varience[i,0] * varience[j,0]
        return covarince_matrix
8 Replies
sora
sora•11mo ago
Could you provide a minimal reproducible example that people can run (include all the code, etc.)?
Harambe
HarambeOP•11mo ago
This is a reduced code sample that reproduces the problem:

message.txt

Michael K
Michael K•11mo ago
This code prints matrix of 3s and then matrix of 9s. No unexpected behaviour on M3 Macbook.
You are using simdwidthof to get a SIMD width specific to your system (mine would be 4) so that may be one reason it doesn't reproduce. Adding bounds checks to your load and store methods would confirm whether you are staying in the allocated pointers.
sora
sora•11mo ago
I can reproduce on my machine (Intel Mac with Docker). Trying to reduce the example further now. Reduced: 
from algorithm import vectorize

struct Vec:
    var length: Int
    var data: DTypePointer[DType.float32]

    fn __init__(inout self, length: Int):
        self.length = length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        for item in range(self.length):
            self.data.store(item, 1)

    fn __copyinit__(inout self, other: Self):
        self.length = other.length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        memcpy(self.data, other.data, self.length)

    fn __del__(owned self):
        self.data.free()

    fn non_deterministic(self) -> Vec:
        let a = Vec(self.length)
        var b = Vec(self.length)
        for i in range(b.length):
            @parameter
            fn f[simd_width: Int](j: Int):
                b.data.store(i, a.data.load(i))
            vectorize[1, f](self.length)
        return b

fn print_vec(self: Vec):
    for i in range(self.length):
        print_no_newline(self.data[i])
        print_no_newline(",")
    print("\n")

fn main():
    let a = Vec(2)
    let b = a.non_deterministic()
    print_vec(b)
from algorithm import vectorize

struct Vec:
    var length: Int
    var data: DTypePointer[DType.float32]

    fn __init__(inout self, length: Int):
        self.length = length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        for item in range(self.length):
            self.data.store(item, 1)

    fn __copyinit__(inout self, other: Self):
        self.length = other.length
        self.data = DTypePointer[DType.float32].alloc(self.length)
        memcpy(self.data, other.data, self.length)

    fn __del__(owned self):
        self.data.free()

    fn non_deterministic(self) -> Vec:
        let a = Vec(self.length)
        var b = Vec(self.length)
        for i in range(b.length):
            @parameter
            fn f[simd_width: Int](j: Int):
                b.data.store(i, a.data.load(i))
            vectorize[1, f](self.length)
        return b

fn print_vec(self: Vec):
    for i in range(self.length):
        print_no_newline(self.data[i])
        print_no_newline(",")
    print("\n")

fn main():
    let a = Vec(2)
    let b = a.non_deterministic()
    print_vec(b)
Michael K
Michael K•11mo ago
For the prospective bug report, the smaller reproducer also shows no bad behaviour on M3 Macbook.
sora
sora•11mo ago
Reported here
GitHub
[BUG]: Code showing non deterministic behaviour · Issue #1755 · mo...
Bug description As title. Doesn't seem to reproduce on M3 Mac. Steps to reproduce from algorithm import vectorize struct Vec: var length: Int var data: DTypePointer[DType.float32] fn init(i...
sora
sora•11mo ago
Seems like a simple problem caused by early destruction, wonder how I missed that. Fix: add _ = a before return b. I think you can fix your CovarienceMatrix similarly.
Harambe
HarambeOP•11mo ago
Updated: The intended fix has solved the problem, thanks for all of the help

Did you find this page helpful?