[Octopus-devel] [Octopus-notify] svn commit: r3861 - trunk/src/h_sys by xavier

[Xavier Andrade]

Hi Florian,

That is preciselly one of the objetives of my changes, currently we do:

do p
  a = <p|psi>
  reduce(a)
  |psi> += a|p>
end do

in the future we should be able to do:

do p
  a(p) = <p|psi>
end do
reduce(a)
do p
  |psi> += a(p)|p>
end do

So we just perform one reduction per hamiltonian application. We could 
even try to parallelize the loops in p using openmp (easy) or atom (state) 
parallelization (probably not a very good idea).

Cheers,

Xavier

On Tue, 11 Mar 2008, Florian Lorenzen wrote:

> Hi Xavier,
>
> I don't know if your commit already helps with that because I know too
> little about the pseudo-part of the code, but the the main bottleneck
> of domain paralellization is not the all-to-all communication in the
> kinetic energy but the allreduce in the application of the
> pseudo-potentials. For 1.2 Million inner points and calculating the
> ground state of a LiH chain the allreduces are 18 % of the entire run
> time and the all-to-all is just 6 %. So, if we want to improve the
> scalabality, the projector code has to be rewritten, ideally just one
> alreduce per Hpsi operation or, even better, just one allreduce per
> block of Kohn-Sham states, but I don't know if that is possible. As
> you seem to be working on that now, perhaps you want to have a look at
> this issue.
>
> Best regards,
>
> Florian
>
> At Mon, 10 Mar 2008 23:13:12 GMT,
> octopus-notify at tddft.org wrote:
>>
>>       Author: xavier
>>         Date: Mon Mar 10 23:13:06 2008
>> New Revision: 3861
>>    Changeset: http://www.tddft.org/trac/octopus/changeset/3861
>>
>> Modified:
>>    trunk/src/h_sys/kb_projector.F90
>>    trunk/src/h_sys/kb_projector_inc.F90
>>    trunk/src/h_sys/projector.F90
>>    trunk/src/h_sys/projector_inc.F90
>>
>> Log:
>> Implemented a new scheme to apply pseudo potentials, for the moment is
>> only for KB.
>>
>>
>> _______________________________________________
>> Octopus-notify mailing list
>> Octopus-notify at tddft.org
>> http://www.tddft.org/mailman/listinfo/octopus-notify
>