Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug Report] ttnn.repeat not working on mesh device #18153

Open
friedrich opened this issue Feb 21, 2025 · 0 comments
Open

[Bug Report] ttnn.repeat not working on mesh device #18153

friedrich opened this issue Feb 21, 2025 · 0 comments
Labels
bug Something isn't working community Stable Diffusion

Comments

@friedrich
Copy link
Contributor

Description

ttnn.repeat fails with the following error on a mesh device due to the fact that ttnn::view does not support mesh devices.

View requires the tensor be stored on device, use reshape instead

Test Code

import ttnn


def test_repeat(*, mesh_device: ttnn.MeshDevice) -> None:
    t = ttnn.ones([32, 32], device=mesh_device)
    ttnn.repeat(t, [2, 1])

Test Output

=================================== FAILURES ===================================
________________________ test_repeat[wormhole_b0-True] _________________________

    def test_repeat(*, mesh_device: ttnn.MeshDevice) -> None:
        t = ttnn.ones([32, 32], device=mesh_device)
>       ttnn.repeat(t, [2, 1])

tests/test_repeat.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = FastOperation(python_fully_qualified_name='ttnn.repeat', function=<ttnn._ttnn.operations.data_movement.repeat_t object...<function default_postprocess_golden_function_outputs at 0x7fea4e591ee0>, is_cpp_operation=True, is_experimental=False)
function_args = (device_id:0
ttnn.Tensor([[ 1.00000,  1.00000,  ...,  1.00000,  1.00000],
             [ 1.00000,  1.00000,  ...,  1.0...00000,  ...,  1.00000,  1.00000]], shape=Shape([32, 32]), dtype=DataType::BFLOAT16, layout=Layout::ROW_MAJOR)
, [2, 1])
function_kwargs = {}

    def __call__(self, *function_args, **function_kwargs):
>       return self.function(*function_args, **function_kwargs)
E       RuntimeError: TT_FATAL @ /home/user/tt-metal/ttnn/cpp/ttnn/operations/data_movement/view/view.cpp:30: ttnn::has_storage_type_of(tensor, ttnn::StorageType::DEVICE)
E       info:
E       View requires the tensor be stored on device, use reshape instead
E       backtrace:
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x10be899) [0x7feb10d95899]
E        --- ttnn::operations::data_movement::ViewOperation::invoke(tt::tt_metal::Tensor const&, tt::tt_metal::Shape const&)
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1289abb) [0x7feb10f60abb]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x128982d) [0x7feb10f6082d]
E        --- ttnn::operations::data_movement::detail::repeat_upper_dims_rm(tt::tt_metal::Tensor const&, unsigned int, unsigned int, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>, tt::tt_metal::MemoryConfig const&)
E        --- ttnn::operations::data_movement::RepeatOperation::invoke(tt::tt_metal::Tensor const&, tt::stl::SmallVector<unsigned int, 8ul> const&, std::__1::optional<tt::tt_metal::MemoryConfig> const&, tt::stl::StrongType<unsigned char, ttnn::QueueIdTag>)
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x129462c) [0x7feb10f6b62c]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1294278) [0x7feb10f6b278]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1294176) [0x7feb10f6b176]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1293b19) [0x7feb10f6ab19]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1293ab4) [0x7feb10f6aab4]
E        --- /home/user/tt-metal/ttnn/ttnn/_ttnn.so(+0x1105f16) [0x7feb10ddcf16]
E        --- /home/user/tt-metal/python_env/bin/python3(PyCFunction_Call+0x59) [0x5f59b9]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyObject_MakeTpCall+0x29e) [0x5f658e]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50adc8]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c85c]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x27e) [0x5f534e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c4ce]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyObject_MakeTpCall+0x29e) [0x5f658e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5e14) [0x570884]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50a59e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c54f]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyObject_MakeTpCall+0x29e) [0x5f658e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x62f5) [0x570d65]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x1b6) [0x5f5d66]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x859) [0x56b2c9]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x1b6) [0x5f5d66]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50a59e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c54f]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x27e) [0x5f534e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x72e) [0x56b19e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50a710]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1882) [0x56c2f2]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x72e) [0x56b19e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1882) [0x56c2f2]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x1b6) [0x5f5d66]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50a59e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c54f]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyObject_MakeTpCall+0x29e) [0x5f658e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x62f5) [0x570d65]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x1b6) [0x5f5d66]
E        --- /home/user/tt-metal/python_env/bin/python3(PyObject_Call+0x1f7) [0x5f52c7]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x1f35) [0x56c9a5]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x50a59e]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalFrameDefault+0x5809) [0x570279]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyEval_EvalCodeWithName+0x26a) [0x56946a]
E        --- /home/user/tt-metal/python_env/bin/python3(_PyFunction_Vectorcall+0x393) [0x5f5f43]
E        --- /home/user/tt-metal/python_env/bin/python3() [0x59c54f]

ttnn/ttnn/decorators.py:333: RuntimeError
=============================== warnings summary ===============================
tests/test_repeat.py::test_repeat[wormhole_b0-True]
  tests/test_repeat.py:13: PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
    def test_repeat(*, mesh_device: ttnn.MeshDevice) -> None:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: /home/user/tt-metal/generated/test_reports/most_recent_tests.xml -
============================== slowest durations ===============================
2.24s setup    tests/test_repeat.py::test_repeat[wormhole_b0-True]
0.00s call     tests/test_repeat.py::test_repeat[wormhole_b0-True]
0.00s teardown tests/test_repeat.py::test_repeat[wormhole_b0-True]
=========================== short test summary info ============================
FAILED tests/test_repeat.py::test_repeat[wormhole_b0-True] - RuntimeError: TT...
========================= 1 failed, 1 warning in 2.35s =========================

Environment

tt-metal: 99e8f45
Platform: Linux-5.4.0-196-generic-x86_64-with-glibc2.29
Python: 3.8.10
@friedrich friedrich added the bug Something isn't working label Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working community Stable Diffusion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@friedrich @zzigler-tt