How can we extract the converged tensors?

[dartsushi]

I played with the tutorials and found out that you could save the resulting tensors by modifying the parameter.general.tensor_save.

The saved data seems to be the combinations of dtype, indices, and bin files, which I found difficult to interpret.
Is it possible to convert it to dat/txt file so that I can analyze it with other languages( like Python or Julia)?

I am sorry if it is written in the document already.
Thanks!

[yomichi]

TeNeS uses mptensor for handling tensors, and so save/load tensors by using mptensor's functionality.

As an example, let's consider a bulk tensor T_ltrbp (indices mean left, top, right, bottom, and physical bonds).
The information file T_0.dat is as follows,

mptensor 0.3.0
matrix_type= 2 (ScaLAPACK)
value_type= 1 (complex)
comm_size= 16
ndim= 5
upper_rank= 3
shape= 10 10 10 10 2
axes_map= 0 1 2 3 4

comm_size is the number of MPI processes (number of distributions).
ndim is the number of indices (legs) of the tensor, and shape is the dimensions of indices (this tensor has 10x10x10x10x2 = 20000 elements).
axes_map is used for lazy evaluation of the transpose operation. When V means the raw data, the i-th index of T is the axes_map[i]-th index of V. (For details, see smorita's comment below)

mptensor stores a tensor as a matrix in order to use LAPACK/ScaLAPACK library.
mptensor combines the first upper_rank indices as the row index of the matrix and considers the remaining indices as the column index (in this case, 1000x20 matrix).
When using ScaLAPACK, this matrix is stored distributedly.

The information on local indices of the distributed matrix for each process is saved in T_0.dat.0000.idx (first process of 16 processes):

local_size= 4096
local_n_row= 256
local_n_col= 16
[global_row]
 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207
 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271
 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335
 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399
 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463
 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527
 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591
 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655
 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719
 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783
 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847
 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911
 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975

[global_col]
 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

This means the first process has a 256x16 local matrix.
global_row says, for example, the 17-th row of the local matrix corresponds to the 65-th row of the global matrix.
Note that mptensor adopts a block-cyclic distribution layout.

The raw data is saved in T_0.dat.0000.bin.
The matrix is stored in column-major (A[1,0] follows A[0,0]).

I will try to write a sample script for loading tensors. Please wait a while.

@smorita
Do you have any corrections and/or comments (particularly on axes_map)?

[smorita]

Sorry for the complicated implementation of axes_map. It always causes confusion, even for me.

axes_map stores an index mapping from a before-transposed tensor to an after-transposed tensor.
Let V a 3-leg tensor with shape=[10, 20, 30] and axes_map=[0, 1, 2]. When we transpose V as

T = V.transpose(Axes(1, 2, 0));

T has shape=[20, 30, 10] and axes_map=[2, 0, 1]. Thus, axes_map satisfies axes_map[iV] = iT.

[dartsushi]

Dear Dr. Motoyama and Dr. Morita,

Thank you very much for your quick response.
I could get how the data is structured.

To open the bin file(where the raw data is stored), which functions in mptensor are related?
There seem to be relevant functions like load_scalapack/load_binary.

[smorita]

The easiest way to open the saved date is to use a member function load of the Tensor class like as

T.load("T_0.dat");

This function automatically redistributes tensor elements even if the number of processes is different when saving and loading.
Behind this load function, io_helper::load_binary function reads a binary file.

[dartsushi]

Thank you very much! I will try and check it!