Modular•8mo ago

pointer to pointer

 
float** ptrs[] = {
        &acts->encoded,

 
float** ptrs[] = {
        &acts->encoded,

acts->encoded is a float* pointer, how to achive this in mojo? I have tried a lot of ways😭

18 Replies

benny•8mo ago

are you looking for DTypePointer[DType.float64]?

zhoujingOP•8mo ago

DTypePointer[DType.float64] is a pointer which points to float64, not a pointer I mean a pointer to pointer like float**

benny•8mo ago

so you wanta pointer to a pointer to a float? Pointer[DTypePointer[DType.float64]]? im unfamiliar with this syntax

zhoujingOP•8mo ago

yes, I can now get acts->encoded as a pointer in mojo, but i don;t know how to point to the got DtypePointer[Dtype.float32] acts->encoded

ModularBot•8mo ago

Congrats @zhoujing, you just advanced to level 1!

zhoujingOP•8mo ago

me too ,haha😆

Martin Dudek•8mo ago

i stuck also there in my attempt to port llm.c - I assume for &params->wte we would need something like Reference(params[].wte) but didnt yet get around my head how to setup a List of References .. good luck I feel better now, kapai.ai just told me: However, please be aware that the use of Reference is still in its early stages and may be awkward to use. 😉 Pointer.address_of. seems to work, at least in this demo:

alias dtype=DType.float32

fn main():

    var sizes = List[Int](3,7,2)

    var total = 0
    for i in range(len(sizes)):
        total += sizes[i]

    var mem = DTypePointer[dtype].alloc(total)

    var a = DTypePointer[dtype]()
    var b = DTypePointer[dtype]()
    var c = DTypePointer[dtype]()

    var ptrs = List(
        Pointer.address_of(a),
        Pointer.address_of(b),
        Pointer.address_of(c)
    )

    var pos = 0
    for i in range(len(ptrs)):
        ptrs[i][] = mem + pos 
        pos += sizes[i]

    b[0] = 17
    print(mem[3])

alias dtype=DType.float32

fn main():

    var sizes = List[Int](3,7,2)

    var total = 0
    for i in range(len(sizes)):
        total += sizes[i]

    var mem = DTypePointer[dtype].alloc(total)

    var a = DTypePointer[dtype]()
    var b = DTypePointer[dtype]()
    var c = DTypePointer[dtype]()

    var ptrs = List(
        Pointer.address_of(a),
        Pointer.address_of(b),
        Pointer.address_of(c)
    )

    var pos = 0
    for i in range(len(ptrs)):
        ptrs[i][] = mem + pos 
        pos += sizes[i]

    b[0] = 17
    print(mem[3])

zhoujingOP•8mo ago

wow, I am porting llm.c too, Thanks for your advice, I will have a try

Martin Dudek•8mo ago

haven't tried it in the llm.mojo, just getting the individual pieces sorted out right now. Looking forward to see both of our solutions 🙏

ModularBot•8mo ago

Congrats @Martin Dudek, you just advanced to level 6!

zhoujingOP•8mo ago

okk,🫰

Martin Dudek•8mo ago

hope you use regexp search replace in your IDE too, eg for (int (\w+) = 0; \1 < (\w+); \1++) { -> for $1 in range($2): 😉

zhoujingOP•8mo ago

I tried a few hours, still a lot work to do🥺 haha

Martin Dudek•8mo ago

i stuck with reading in the "gpt2_124M.bin" file - gpt2_build_from_checkpoint did you get this sorted already?

zhoujingOP•8mo ago

I got stuck in this place

typedef struct {
    // hyperparameters
    int B;
    int T;
    // input handling and its state
    FILE* tokens_file;
    long file_size;
    long current_position;
    // output memory
    int* batch;
    int* inputs;
    int* targets;
    // convenience variables
    int num_batches;
} DataLoader;

typedef struct {
    // hyperparameters
    int B;
    int T;
    // input handling and its state
    FILE* tokens_file;
    long file_size;
    long current_position;
    // output memory
    int* batch;
    int* inputs;
    int* targets;
    // convenience variables
    int num_batches;
} DataLoader;

the FILE pointer I am downloading data now. I will try

Martin Dudek•8mo ago

in python this works:

import struct
import sys

with open("gpt2_124M.bin", 'rb') as file:
   
    model_header_data = file.read(256 * 4)
    model_header = struct.unpack('256i', model_header_data)

    # Check the magic number and version
    if model_header[0] != 20240326:
        print("Bad magic model file")
        sys.exit(1)
    if model_header[1] != 1:
        print("Bad version in model file")
        sys.exit(1)

    print(model_header[0])

import struct
import sys

with open("gpt2_124M.bin", 'rb') as file:
   
    model_header_data = file.read(256 * 4)
    model_header = struct.unpack('256i', model_header_data)

    # Check the magic number and version
    if model_header[0] != 20240326:
        print("Bad magic model file")
        sys.exit(1)
    if model_header[1] != 1:
        print("Bad version in model file")
        sys.exit(1)

    print(model_header[0])

but i dont know how t get this done in Mojo, struct.unpack does some magic it seems

zhoujingOP•8mo ago

I got stuck here too....

Martin Dudek•8mo ago

Found a solution here https://github.com/tairov/llama2.mojo/blob/master/llama2.mojo just need to use read instead of read_byte . Not sure why this works now and my atempts not but ....

var checkpoint_path = "gpt2_124M.bin"

var f = open(checkpoint_path, "r")
# reading 7 vars of type DType.int32 from the file
var bytes_of_config_params = 256 * sizeof[DType.int32]()
# config_data_raw id Tensor[DType.int8] with bytes_of_config_params elements
var config_data_raw = f.read(bytes_of_config_params)
f.close()

var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()
var config_data = Tensor[DType.int32](int32_ptr, 256)

print(config_data[0])

var checkpoint_path = "gpt2_124M.bin"

var f = open(checkpoint_path, "r")
# reading 7 vars of type DType.int32 from the file
var bytes_of_config_params = 256 * sizeof[DType.int32]()
# config_data_raw id Tensor[DType.int8] with bytes_of_config_params elements
var config_data_raw = f.read(bytes_of_config_params)
f.close()

var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()
var config_data = Tensor[DType.int32](int32_ptr, 256)

print(config_data[0])

GitHub

llama2.mojo/llama2.mojo at master · tairov/llama2.mojo

Inference Llama 2 in one file of pure 🔥. Contribute to tairov/llama2.mojo development by creating an account on GitHub.

Gaming

Programming

pointer to pointer