Incorrect calculation of cosine similarity

[Alex-EEE]

clip.cpp does this

float clip_similarity_score(const float * vec1, const float * vec2, const int vec_dim) {
    float dot_product = 0.0;
    for (int i = 0; i < vec_dim; i++) {
        dot_product += vec1[i] * vec2[i];
    }

    return dot_product;
}

But this is not the cosine similarity. In fact, you're just calculating the dot product, which won't behave the same as CLIP was trained (though you can probably get lucky and see some OK softmaxes)

You need to normalize all vectors before the dot product