Skip to content

Commit

Permalink
runtest expand
Browse files Browse the repository at this point in the history
  • Loading branch information
@billera
billera committed Jan 31, 2025
1 parent 7a2f1a2 commit 0c8b88e
Showing 1 changed file with 31 additions and 0 deletions.
31 changes: 31 additions & 0 deletions test/runtests.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,37 @@ using ChainRulesTestUtils

@testset "InvariantPointAttention.jl" begin

@testset "Rope Expand" begin
dims = 8
settings = IPA_settings(dims)

# generate random data
L = 10
R = 10
B = 1
siL = randn(Float32, dims, L, B)
siR = siL
#zij = randn(Float32, c_z, R, L, B)
TiL = (get_rotation(L, B), get_translation(L, B))
TiR = TiL

# Left and right equal for self attention
TiL == TiR
siL == siR

# Extend the cache along both left and right
ipa = IPCrossA(settings)
cache = InvariantPointAttention.IPACache(settings, B)

rope = IPARoPE(ipa.settings.c, 100)
siRs = []
for i in 1:10
si, cache = InvariantPointAttention.expand(ipa, cache, TiL, siL, 1, TiR, siR, 1, rope= rope.rope[i:i])
push!(siRs, si)
end
cat(siRs..., dims = 2) ≈ ipa(TiL, siL, TiR, siR; mask = right_to_left_mask(10), rope = rope[1:10])
end

@testset "IPAsoftmax_invariance" begin
batch_size = 3
framesL = 100
Expand Down

2 comments on commit 0c8b88e

@billera
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/124078

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.1.5 -m "<description of version>" 0c8b88efee5ff3f0bf98e3bb1885c44313bd2184
git push origin v0.1.5

Also, note the warning: Version 0.1.5 skips over 0.1.3
This can be safely ignored. However, if you want to fix this you can do so. Call register() again after making the fix. This will update the Pull request.

Please sign in to comment.