Apache Ignite RPM packaging (Stage II)
Locked
12
12
 
|
Hi, Igniters!
Release 2.4 is almost there, at least binary part of it, so I'd like to move forward to further improve and widen AI delivery through packages. As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to improve usability and significantly reduce required download sizes, I purpose that in 2.5 release we introduce splitted delivery as follows: - CORE - bin - config - libs (!optional) - OPTIONAL LIBS - BENCHMARKS - DOCS (?) - EXAMPLES - .NET PLATFORM FILES - C++ PLATFORM FILES This architecture, as I assume, will add flexibility (no reason to download all 280M+ of binaries where you are to run only core node functionality) and maintainability (you are in full control of what is installed on your system). After successful architecture choice, same scheme are planned to be used in DEB packages as well. WDYT? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ |
Re: Apache Ignite RPM packaging (Stage II)
|
Hi,
Did not understand fully that splitted delivery means. Is it correct,that user will have install install ignite:core install ignite:opt_libs with separated commands? Or there will be some aggregating package like ignite:full and several options like ignite:core. Sincerely, Dmitriy Pavlov ср, 14 мар. 2018 г. в 12:36, vveider <[hidden email]>: > Hi, Igniters! > > > Release 2.4 is almost there, at least binary part of it, so I'd like to > move > forward to further improve and widen AI delivery through packages. > As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to > improve usability and significantly reduce required download sizes, I > purpose that in 2.5 release we introduce splitted delivery as follows: > - CORE > - bin > - config > - libs (!optional) > - OPTIONAL LIBS > - BENCHMARKS > - DOCS (?) > - EXAMPLES > - .NET PLATFORM FILES > - C++ PLATFORM FILES > > This architecture, as I assume, will add flexibility (no reason to download > all 280M+ of binaries where you are to run only core node functionality) > and > maintainability (you are in full control of what is installed on your > system). > > After successful architecture choice, same scheme are planned to be used in > DEB packages as well. > > WDYT? > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
|
|
> On 14 Mar 2018, at 12:46, Dmitry Pavlov <[hidden email]> wrote: > > Hi, > > Did not understand fully that splitted delivery means. > Is it correct,that user will have install > install ignite:core > install ignite:opt_libs > with separated commands? It will be one command with several arguments, i.e. yum install ignite-core ignite-optional Also, I considered splitting optional libs per package (like php / perl / etc. do, for example) but not sure whether these efforts are appropriate at this moment. I guess this scheme must have huge support from the community. > > Or there will be some aggregating package like ignite:full and several > options like ignite:core. Nice idea! If community will agree, I can add aggregating virtual package with dependencies on all other for corresponding release. It can even keep current name ‘apache-ignite’ :) > > Sincerely, > Dmitriy Pavlov > > ср, 14 мар. 2018 г. в 12:36, vveider <[hidden email]>: > >> Hi, Igniters! >> >> >> Release 2.4 is almost there, at least binary part of it, so I'd like to >> move >> forward to further improve and widen AI delivery through packages. >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to >> improve usability and significantly reduce required download sizes, I >> purpose that in 2.5 release we introduce splitted delivery as follows: >> - CORE >> - bin >> - config >> - libs (!optional) >> - OPTIONAL LIBS >> - BENCHMARKS >> - DOCS (?) >> - EXAMPLES >> - .NET PLATFORM FILES >> - C++ PLATFORM FILES >> >> This architecture, as I assume, will add flexibility (no reason to download >> all 280M+ of binaries where you are to run only core node functionality) >> and >> maintainability (you are in full control of what is installed on your >> system). >> >> After successful architecture choice, same scheme are planned to be used in >> DEB packages as well. >> >> WDYT? >> >> >> >> -- >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >> |
|
In reply to this post by vveider
Petr,
How big would be a final core module? As for the optional libs do you suggest installing them with a single command or each module can be installed separately? -- Denis On Wed, Mar 14, 2018 at 2:36 AM, vveider <[hidden email]> wrote: > Hi, Igniters! > > > Release 2.4 is almost there, at least binary part of it, so I'd like to > move > forward to further improve and widen AI delivery through packages. > As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to > improve usability and significantly reduce required download sizes, I > purpose that in 2.5 release we introduce splitted delivery as follows: > - CORE > - bin > - config > - libs (!optional) > - OPTIONAL LIBS > - BENCHMARKS > - DOCS (?) > - EXAMPLES > - .NET PLATFORM FILES > - C++ PLATFORM FILES > > This architecture, as I assume, will add flexibility (no reason to download > all 280M+ of binaries where you are to run only core node functionality) > and > maintainability (you are in full control of what is installed on your > system). > > After successful architecture choice, same scheme are planned to be used in > DEB packages as well. > > WDYT? > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
Re: Apache Ignite RPM packaging (Stage II)
|
In reply to this post by vveider
Peter,
I don't think the package size of 280M is going to be a problem at all, but what you are suggesting can be an improvement down the road. In the mean time, I think our top priority should be to provide packages for Debian and Ubuntu. Having only RPMs is not nearly enough. Agree? D. On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: > Hi, Igniters! > > > Release 2.4 is almost there, at least binary part of it, so I'd like to > move > forward to further improve and widen AI delivery through packages. > As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to > improve usability and significantly reduce required download sizes, I > purpose that in 2.5 release we introduce splitted delivery as follows: > - CORE > - bin > - config > - libs (!optional) > - OPTIONAL LIBS > - BENCHMARKS > - DOCS (?) > - EXAMPLES > - .NET PLATFORM FILES > - C++ PLATFORM FILES > > This architecture, as I assume, will add flexibility (no reason to download > all 280M+ of binaries where you are to run only core node functionality) > and > maintainability (you are in full control of what is installed on your > system). > > After successful architecture choice, same scheme are planned to be used in > DEB packages as well. > > WDYT? > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
|
|
In reply to this post by dmagda
> On 14 Mar 2018, at 22:12, Denis Magda <[hidden email]> wrote: > > Petr, > > How big would be a final core module? Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own package. > As for the optional libs do you > suggest installing them with a single command or each module can be > installed separately? Eventually — separately. If user wishes to try some new integration or lib, there is no need to download all of them. Also it will be clearer what new integrations / libs we add (if we add) each release. > > -- > Denis > > On Wed, Mar 14, 2018 at 2:36 AM, vveider <[hidden email]> wrote: > >> Hi, Igniters! >> >> >> Release 2.4 is almost there, at least binary part of it, so I'd like to >> move >> forward to further improve and widen AI delivery through packages. >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to >> improve usability and significantly reduce required download sizes, I >> purpose that in 2.5 release we introduce splitted delivery as follows: >> - CORE >> - bin >> - config >> - libs (!optional) >> - OPTIONAL LIBS >> - BENCHMARKS >> - DOCS (?) >> - EXAMPLES >> - .NET PLATFORM FILES >> - C++ PLATFORM FILES >> >> This architecture, as I assume, will add flexibility (no reason to download >> all 280M+ of binaries where you are to run only core node functionality) >> and >> maintainability (you are in full control of what is installed on your >> system). >> >> After successful architecture choice, same scheme are planned to be used in >> DEB packages as well. >> >> WDYT? >> >> >> >> -- >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >> |
|
|
In reply to this post by dsetrakyan
Not a problem, rather nuisance. Also, when we will move to official repositories, there can be a problem from OS community.
Concerning DEB packages — I plan to use RPM as base for DEB package build (package layout / install scripts) for speeding up things and excluding possible duplication and desynchronisation, so its a matter of ’sit and do’ rather then some technical research. Thats why I rose discussion about future package architecture, so that after agreement I'm be able to pack both RPM and DEB identically. Yet, if you insist, I can create DEB package according to current RPM layout in no time. > On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> wrote: > > Peter, > > I don't think the package size of 280M is going to be a problem at all, but > what you are suggesting can be an improvement down the road. > > In the mean time, I think our top priority should be to provide packages > for Debian and Ubuntu. Having only RPMs is not nearly enough. > > Agree? > > D. > > On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: > >> Hi, Igniters! >> >> >> Release 2.4 is almost there, at least binary part of it, so I'd like to >> move >> forward to further improve and widen AI delivery through packages. >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to >> improve usability and significantly reduce required download sizes, I >> purpose that in 2.5 release we introduce splitted delivery as follows: >> - CORE >> - bin >> - config >> - libs (!optional) >> - OPTIONAL LIBS >> - BENCHMARKS >> - DOCS (?) >> - EXAMPLES >> - .NET PLATFORM FILES >> - C++ PLATFORM FILES >> >> This architecture, as I assume, will add flexibility (no reason to download >> all 280M+ of binaries where you are to run only core node functionality) >> and >> maintainability (you are in full control of what is installed on your >> system). >> >> After successful architecture choice, same scheme are planned to be used in >> DEB packages as well. >> >> WDYT? >> >> >> >> -- >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >> |
Re: Apache Ignite RPM packaging (Stage II)
|
|
Will Debian package work for Ubuntu?
D. On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> wrote: > Not a problem, rather nuisance. Also, when we will move to official > repositories, there can be a problem from OS community. > > Concerning DEB packages — I plan to use RPM as base for DEB package build > (package layout / install scripts) for speeding up things and excluding > possible duplication and desynchronisation, so its a matter of ’sit and do’ > rather then some technical research. Thats why I rose discussion about > future package architecture, so that after agreement I'm be able to pack > both RPM and DEB identically. > > Yet, if you insist, I can create DEB package according to current RPM > layout in no time. > > > > > On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> > wrote: > > > > Peter, > > > > I don't think the package size of 280M is going to be a problem at all, > but > > what you are suggesting can be an improvement down the road. > > > > In the mean time, I think our top priority should be to provide packages > > for Debian and Ubuntu. Having only RPMs is not nearly enough. > > > > Agree? > > > > D. > > > > On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: > > > >> Hi, Igniters! > >> > >> > >> Release 2.4 is almost there, at least binary part of it, so I'd like to > >> move > >> forward to further improve and widen AI delivery through packages. > >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, > to > >> improve usability and significantly reduce required download sizes, I > >> purpose that in 2.5 release we introduce splitted delivery as follows: > >> - CORE > >> - bin > >> - config > >> - libs (!optional) > >> - OPTIONAL LIBS > >> - BENCHMARKS > >> - DOCS (?) > >> - EXAMPLES > >> - .NET PLATFORM FILES > >> - C++ PLATFORM FILES > >> > >> This architecture, as I assume, will add flexibility (no reason to > download > >> all 280M+ of binaries where you are to run only core node functionality) > >> and > >> maintainability (you are in full control of what is installed on your > >> system). > >> > >> After successful architecture choice, same scheme are planned to be > used in > >> DEB packages as well. > >> > >> WDYT? > >> > >> > >> > >> -- > >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > >> > > |
|
|
Considering that DEV package for now is almost platform independent (its a java application more or less), that package will work almost on any DEB-based linux, including but not limited to Ubuntu, Debian, etc.
The only restriction is existence of systemctl (systemd) service manager — we are dependent on it. Thats why, for instance, our RPM repository is called simply ‘rpm’ and package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, etc. with presence of aforementioned systemd. > On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> wrote: > > Will Debian package work for Ubuntu? > > D. > > On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> wrote: > >> Not a problem, rather nuisance. Also, when we will move to official >> repositories, there can be a problem from OS community. >> >> Concerning DEB packages — I plan to use RPM as base for DEB package build >> (package layout / install scripts) for speeding up things and excluding >> possible duplication and desynchronisation, so its a matter of ’sit and do’ >> rather then some technical research. Thats why I rose discussion about >> future package architecture, so that after agreement I'm be able to pack >> both RPM and DEB identically. >> >> Yet, if you insist, I can create DEB package according to current RPM >> layout in no time. >> >> >> >>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> >> wrote: >>> >>> Peter, >>> >>> I don't think the package size of 280M is going to be a problem at all, >> but >>> what you are suggesting can be an improvement down the road. >>> >>> In the mean time, I think our top priority should be to provide packages >>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>> >>> Agree? >>> >>> D. >>> >>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: >>> >>>> Hi, Igniters! >>>> >>>> >>>> Release 2.4 is almost there, at least binary part of it, so I'd like to >>>> move >>>> forward to further improve and widen AI delivery through packages. >>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, >> to >>>> improve usability and significantly reduce required download sizes, I >>>> purpose that in 2.5 release we introduce splitted delivery as follows: >>>> - CORE >>>> - bin >>>> - config >>>> - libs (!optional) >>>> - OPTIONAL LIBS >>>> - BENCHMARKS >>>> - DOCS (?) >>>> - EXAMPLES >>>> - .NET PLATFORM FILES >>>> - C++ PLATFORM FILES >>>> >>>> This architecture, as I assume, will add flexibility (no reason to >> download >>>> all 280M+ of binaries where you are to run only core node functionality) >>>> and >>>> maintainability (you are in full control of what is installed on your >>>> system). >>>> >>>> After successful architecture choice, same scheme are planned to be >> used in >>>> DEB packages as well. >>>> >>>> WDYT? >>>> >>>> >>>> >>>> -- >>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >>>> >> >> |
|
|
*DEB package
> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: > > Considering that DEV package for now is almost platform independent (its a java application more or less), that package will work almost on any DEB-based linux, including but not limited to Ubuntu, Debian, etc. > The only restriction is existence of systemctl (systemd) service manager — we are dependent on it. > > Thats why, for instance, our RPM repository is called simply ‘rpm’ and package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, etc. with presence of aforementioned systemd. > > > >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> wrote: >> >> Will Debian package work for Ubuntu? >> >> D. >> >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> wrote: >> >>> Not a problem, rather nuisance. Also, when we will move to official >>> repositories, there can be a problem from OS community. >>> >>> Concerning DEB packages — I plan to use RPM as base for DEB package build >>> (package layout / install scripts) for speeding up things and excluding >>> possible duplication and desynchronisation, so its a matter of ’sit and do’ >>> rather then some technical research. Thats why I rose discussion about >>> future package architecture, so that after agreement I'm be able to pack >>> both RPM and DEB identically. >>> >>> Yet, if you insist, I can create DEB package according to current RPM >>> layout in no time. >>> >>> >>> >>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> >>> wrote: >>>> >>>> Peter, >>>> >>>> I don't think the package size of 280M is going to be a problem at all, >>> but >>>> what you are suggesting can be an improvement down the road. >>>> >>>> In the mean time, I think our top priority should be to provide packages >>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>>> >>>> Agree? >>>> >>>> D. >>>> >>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: >>>> >>>>> Hi, Igniters! >>>>> >>>>> >>>>> Release 2.4 is almost there, at least binary part of it, so I'd like to >>>>> move >>>>> forward to further improve and widen AI delivery through packages. >>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, >>> to >>>>> improve usability and significantly reduce required download sizes, I >>>>> purpose that in 2.5 release we introduce splitted delivery as follows: >>>>> - CORE >>>>> - bin >>>>> - config >>>>> - libs (!optional) >>>>> - OPTIONAL LIBS >>>>> - BENCHMARKS >>>>> - DOCS (?) >>>>> - EXAMPLES >>>>> - .NET PLATFORM FILES >>>>> - C++ PLATFORM FILES >>>>> >>>>> This architecture, as I assume, will add flexibility (no reason to >>> download >>>>> all 280M+ of binaries where you are to run only core node functionality) >>>>> and >>>>> maintainability (you are in full control of what is installed on your >>>>> system). >>>>> >>>>> After successful architecture choice, same scheme are planned to be >>> used in >>>>> DEB packages as well. >>>>> >>>>> WDYT? >>>>> >>>>> >>>>> >>>>> -- >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >>>>> >>> >>> > |
|
|
>
> > > > How big would be a final core module? > Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own > package. Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and propose the simplest modular system: - core module that includes basic Ignite capabilities including SQL, compute grid, service grid, k/v - optional module hosts the rest - ML, streamers integration (kafka, flink), kubernetes, etc. What do you think? -- Denis On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> wrote: > *DEB package > > > > On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: > > > > Considering that DEV package for now is almost platform independent (its > a java application more or less), that package will work almost on any > DEB-based linux, including but not limited to Ubuntu, Debian, etc. > > The only restriction is existence of systemctl (systemd) service manager > — we are dependent on it. > > > > Thats why, for instance, our RPM repository is called simply ‘rpm’ and > package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, > etc. with presence of aforementioned systemd. > > > > > > > >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> > wrote: > >> > >> Will Debian package work for Ubuntu? > >> > >> D. > >> > >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> > wrote: > >> > >>> Not a problem, rather nuisance. Also, when we will move to official > >>> repositories, there can be a problem from OS community. > >>> > >>> Concerning DEB packages — I plan to use RPM as base for DEB package > build > >>> (package layout / install scripts) for speeding up things and excluding > >>> possible duplication and desynchronisation, so its a matter of ’sit > and do’ > >>> rather then some technical research. Thats why I rose discussion about > >>> future package architecture, so that after agreement I'm be able to > pack > >>> both RPM and DEB identically. > >>> > >>> Yet, if you insist, I can create DEB package according to current RPM > >>> layout in no time. > >>> > >>> > >>> > >>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> > >>> wrote: > >>>> > >>>> Peter, > >>>> > >>>> I don't think the package size of 280M is going to be a problem at > all, > >>> but > >>>> what you are suggesting can be an improvement down the road. > >>>> > >>>> In the mean time, I think our top priority should be to provide > packages > >>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. > >>>> > >>>> Agree? > >>>> > >>>> D. > >>>> > >>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> wrote: > >>>> > >>>>> Hi, Igniters! > >>>>> > >>>>> > >>>>> Release 2.4 is almost there, at least binary part of it, so I'd like > to > >>>>> move > >>>>> forward to further improve and widen AI delivery through packages. > >>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ > and, > >>> to > >>>>> improve usability and significantly reduce required download sizes, I > >>>>> purpose that in 2.5 release we introduce splitted delivery as > follows: > >>>>> - CORE > >>>>> - bin > >>>>> - config > >>>>> - libs (!optional) > >>>>> - OPTIONAL LIBS > >>>>> - BENCHMARKS > >>>>> - DOCS (?) > >>>>> - EXAMPLES > >>>>> - .NET PLATFORM FILES > >>>>> - C++ PLATFORM FILES > >>>>> > >>>>> This architecture, as I assume, will add flexibility (no reason to > >>> download > >>>>> all 280M+ of binaries where you are to run only core node > functionality) > >>>>> and > >>>>> maintainability (you are in full control of what is installed on your > >>>>> system). > >>>>> > >>>>> After successful architecture choice, same scheme are planned to be > >>> used in > >>>>> DEB packages as well. > >>>>> > >>>>> WDYT? > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > >>>>> > >>> > >>> > > > > |
Re: Apache Ignite RPM packaging (Stage II)
|
|
I like idea of keeping simple system of modules, so +1 from me.
Where optional libs (e.g Direct IO plugin) would be included, would it be core or optional? чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: > > > > > > > > How big would be a final core module? > > Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own > > package. > > > Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and > propose the simplest modular system: > > - core module that includes basic Ignite capabilities including SQL, > compute grid, service grid, k/v > - optional module hosts the rest - ML, streamers integration (kafka, > flink), kubernetes, etc. > > What do you think? > > -- > Denis > > On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> wrote: > > > *DEB package > > > > > > > On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: > > > > > > Considering that DEV package for now is almost platform independent > (its > > a java application more or less), that package will work almost on any > > DEB-based linux, including but not limited to Ubuntu, Debian, etc. > > > The only restriction is existence of systemctl (systemd) service > manager > > — we are dependent on it. > > > > > > Thats why, for instance, our RPM repository is called simply ‘rpm’ and > > package has no arch or dist suffix — it will work on CentOS, RHEL, > Fedora, > > etc. with presence of aforementioned systemd. > > > > > > > > > > > >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> > > wrote: > > >> > > >> Will Debian package work for Ubuntu? > > >> > > >> D. > > >> > > >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> > > wrote: > > >> > > >>> Not a problem, rather nuisance. Also, when we will move to official > > >>> repositories, there can be a problem from OS community. > > >>> > > >>> Concerning DEB packages — I plan to use RPM as base for DEB package > > build > > >>> (package layout / install scripts) for speeding up things and > excluding > > >>> possible duplication and desynchronisation, so its a matter of ’sit > > and do’ > > >>> rather then some technical research. Thats why I rose discussion > about > > >>> future package architecture, so that after agreement I'm be able to > > pack > > >>> both RPM and DEB identically. > > >>> > > >>> Yet, if you insist, I can create DEB package according to current RPM > > >>> layout in no time. > > >>> > > >>> > > >>> > > >>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> > > >>> wrote: > > >>>> > > >>>> Peter, > > >>>> > > >>>> I don't think the package size of 280M is going to be a problem at > > all, > > >>> but > > >>>> what you are suggesting can be an improvement down the road. > > >>>> > > >>>> In the mean time, I think our top priority should be to provide > > packages > > >>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. > > >>>> > > >>>> Agree? > > >>>> > > >>>> D. > > >>>> > > >>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> > wrote: > > >>>> > > >>>>> Hi, Igniters! > > >>>>> > > >>>>> > > >>>>> Release 2.4 is almost there, at least binary part of it, so I'd > like > > to > > >>>>> move > > >>>>> forward to further improve and widen AI delivery through packages. > > >>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ > > and, > > >>> to > > >>>>> improve usability and significantly reduce required download > sizes, I > > >>>>> purpose that in 2.5 release we introduce splitted delivery as > > follows: > > >>>>> - CORE > > >>>>> - bin > > >>>>> - config > > >>>>> - libs (!optional) > > >>>>> - OPTIONAL LIBS > > >>>>> - BENCHMARKS > > >>>>> - DOCS (?) > > >>>>> - EXAMPLES > > >>>>> - .NET PLATFORM FILES > > >>>>> - C++ PLATFORM FILES > > >>>>> > > >>>>> This architecture, as I assume, will add flexibility (no reason to > > >>> download > > >>>>> all 280M+ of binaries where you are to run only core node > > functionality) > > >>>>> and > > >>>>> maintainability (you are in full control of what is installed on > your > > >>>>> system). > > >>>>> > > >>>>> After successful architecture choice, same scheme are planned to be > > >>> used in > > >>>>> DEB packages as well. > > >>>>> > > >>>>> WDYT? > > >>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > >>>>> > > >>> > > >>> > > > > > > > > |
|
|
I suppose that most everything if not all from libs/options will go to OPTIONAL (I’d call it simply ‘apache-ignite-libs').
More precise lib selection (if something from optional would better to have in core package) will be discussed right after preliminary split architecture agreement. > On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> wrote: > > I like idea of keeping simple system of modules, so +1 from me. > > Where optional libs (e.g Direct IO plugin) would be included, would it be > core or optional? > > чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: > >>> >>>> >>>> How big would be a final core module? >>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own >>> package. >> >> >> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and >> propose the simplest modular system: >> >> - core module that includes basic Ignite capabilities including SQL, >> compute grid, service grid, k/v >> - optional module hosts the rest - ML, streamers integration (kafka, >> flink), kubernetes, etc. >> >> What do you think? >> >> -- >> Denis >> >> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> wrote: >> >>> *DEB package >>> >>> >>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: >>>> >>>> Considering that DEV package for now is almost platform independent >> (its >>> a java application more or less), that package will work almost on any >>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >>>> The only restriction is existence of systemctl (systemd) service >> manager >>> — we are dependent on it. >>>> >>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ and >>> package has no arch or dist suffix — it will work on CentOS, RHEL, >> Fedora, >>> etc. with presence of aforementioned systemd. >>>> >>>> >>>> >>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> >>> wrote: >>>>> >>>>> Will Debian package work for Ubuntu? >>>>> >>>>> D. >>>>> >>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> >>> wrote: >>>>> >>>>>> Not a problem, rather nuisance. Also, when we will move to official >>>>>> repositories, there can be a problem from OS community. >>>>>> >>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package >>> build >>>>>> (package layout / install scripts) for speeding up things and >> excluding >>>>>> possible duplication and desynchronisation, so its a matter of ’sit >>> and do’ >>>>>> rather then some technical research. Thats why I rose discussion >> about >>>>>> future package architecture, so that after agreement I'm be able to >>> pack >>>>>> both RPM and DEB identically. >>>>>> >>>>>> Yet, if you insist, I can create DEB package according to current RPM >>>>>> layout in no time. >>>>>> >>>>>> >>>>>> >>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> >>>>>> wrote: >>>>>>> >>>>>>> Peter, >>>>>>> >>>>>>> I don't think the package size of 280M is going to be a problem at >>> all, >>>>>> but >>>>>>> what you are suggesting can be an improvement down the road. >>>>>>> >>>>>>> In the mean time, I think our top priority should be to provide >>> packages >>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>>>>>> >>>>>>> Agree? >>>>>>> >>>>>>> D. >>>>>>> >>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> >> wrote: >>>>>>> >>>>>>>> Hi, Igniters! >>>>>>>> >>>>>>>> >>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >> like >>> to >>>>>>>> move >>>>>>>> forward to further improve and widen AI delivery through packages. >>>>>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ >>> and, >>>>>> to >>>>>>>> improve usability and significantly reduce required download >> sizes, I >>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >>> follows: >>>>>>>> - CORE >>>>>>>> - bin >>>>>>>> - config >>>>>>>> - libs (!optional) >>>>>>>> - OPTIONAL LIBS >>>>>>>> - BENCHMARKS >>>>>>>> - DOCS (?) >>>>>>>> - EXAMPLES >>>>>>>> - .NET PLATFORM FILES >>>>>>>> - C++ PLATFORM FILES >>>>>>>> >>>>>>>> This architecture, as I assume, will add flexibility (no reason to >>>>>> download >>>>>>>> all 280M+ of binaries where you are to run only core node >>> functionality) >>>>>>>> and >>>>>>>> maintainability (you are in full control of what is installed on >> your >>>>>>>> system). >>>>>>>> >>>>>>>> After successful architecture choice, same scheme are planned to be >>>>>> used in >>>>>>>> DEB packages as well. >>>>>>>> >>>>>>>> WDYT? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >>>>>>>> >>>>>> >>>>>> >>>> >>> >>> >> |
|
|
Hi, Igniters!
Here are some news on our RPM packages initiative. 1. I’ve finished preliminary developing of Stage II version of RPM packages [1]. Main “new feature” is — split design. Also I’ve added package.sh script for automating package building process which will help organise corresponding builds in TC as well as simplify process for developers who wishes to have custom packages. PR#3703 [2] is ready for review. Denis, in order to catch up with Apache Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] repositories on Apache Bintray. Though they are already prepared for hosting RPM and DEB packages respectively, and there is a way of linking them to apache.org/dist/ignite page, there is possible alternative in storing there only plain directory layout corresponding to each repository type (RPM and DEB) and manage this layout (repodata, distributions, versions, etc.) by ourselves, having more control over repositories but lacking some simplicity of deploying new releases. WDYT? Should we try Cassandra approach? They are storing their DEB packages as I described above [5]. Also — a question arose while I was working on this issue: which OSes (and which versions of each) are we going to support (if we are going) in terms of step-by-step list? Currently RPM packages are tested only with latest CentOS (and, respectively — RHEL), but there are a lot more RPM-based distributives [6] some of which are more o less popular among OS community (ALT, Fedora, openSUSE, etc.). [1] https://issues.apache.org/jira/browse/IGNITE-7647 [2] https://github.com/apache/ignite/pull/3703 [3] https://bintray.com/apache/ignite-rpm [4] https://bintray.com/apache/ignite-deb [5] https://bintray.com/apache/cassandra/debian#files/ [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions > On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: > > I suppose that most everything if not all from libs/options will go to OPTIONAL (I’d call it simply ‘apache-ignite-libs'). > More precise lib selection (if something from optional would better to have in core package) will be discussed right after preliminary split architecture agreement. > > > >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> wrote: >> >> I like idea of keeping simple system of modules, so +1 from me. >> >> Where optional libs (e.g Direct IO plugin) would be included, would it be >> core or optional? >> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: >> >>>> >>>>> >>>>> How big would be a final core module? >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own >>>> package. >>> >>> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and >>> propose the simplest modular system: >>> >>> - core module that includes basic Ignite capabilities including SQL, >>> compute grid, service grid, k/v >>> - optional module hosts the rest - ML, streamers integration (kafka, >>> flink), kubernetes, etc. >>> >>> What do you think? >>> >>> -- >>> Denis >>> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> wrote: >>> >>>> *DEB package >>>> >>>> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: >>>>> >>>>> Considering that DEV package for now is almost platform independent >>> (its >>>> a java application more or less), that package will work almost on any >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >>>>> The only restriction is existence of systemctl (systemd) service >>> manager >>>> — we are dependent on it. >>>>> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ and >>>> package has no arch or dist suffix — it will work on CentOS, RHEL, >>> Fedora, >>>> etc. with presence of aforementioned systemd. >>>>> >>>>> >>>>> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> >>>> wrote: >>>>>> >>>>>> Will Debian package work for Ubuntu? >>>>>> >>>>>> D. >>>>>> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> >>>> wrote: >>>>>> >>>>>>> Not a problem, rather nuisance. Also, when we will move to official >>>>>>> repositories, there can be a problem from OS community. >>>>>>> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package >>>> build >>>>>>> (package layout / install scripts) for speeding up things and >>> excluding >>>>>>> possible duplication and desynchronisation, so its a matter of ’sit >>>> and do’ >>>>>>> rather then some technical research. Thats why I rose discussion >>> about >>>>>>> future package architecture, so that after agreement I'm be able to >>>> pack >>>>>>> both RPM and DEB identically. >>>>>>> >>>>>>> Yet, if you insist, I can create DEB package according to current RPM >>>>>>> layout in no time. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <[hidden email]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Peter, >>>>>>>> >>>>>>>> I don't think the package size of 280M is going to be a problem at >>>> all, >>>>>>> but >>>>>>>> what you are suggesting can be an improvement down the road. >>>>>>>> >>>>>>>> In the mean time, I think our top priority should be to provide >>>> packages >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>>>>>>> >>>>>>>> Agree? >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> >>> wrote: >>>>>>>> >>>>>>>>> Hi, Igniters! >>>>>>>>> >>>>>>>>> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >>> like >>>> to >>>>>>>>> move >>>>>>>>> forward to further improve and widen AI delivery through packages. >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ >>>> and, >>>>>>> to >>>>>>>>> improve usability and significantly reduce required download >>> sizes, I >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >>>> follows: >>>>>>>>> - CORE >>>>>>>>> - bin >>>>>>>>> - config >>>>>>>>> - libs (!optional) >>>>>>>>> - OPTIONAL LIBS >>>>>>>>> - BENCHMARKS >>>>>>>>> - DOCS (?) >>>>>>>>> - EXAMPLES >>>>>>>>> - .NET PLATFORM FILES >>>>>>>>> - C++ PLATFORM FILES >>>>>>>>> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason to >>>>>>> download >>>>>>>>> all 280M+ of binaries where you are to run only core node >>>> functionality) >>>>>>>>> and >>>>>>>>> maintainability (you are in full control of what is installed on >>> your >>>>>>>>> system). >>>>>>>>> >>>>>>>>> After successful architecture choice, same scheme are planned to be >>>>>>> used in >>>>>>>>> DEB packages as well. >>>>>>>>> >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>>> >>> > |
Re: Apache Ignite RPM packaging (Stage II)
|
|
Petr,
I am confused. Do we already have Debian packages? D. On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <[hidden email]> wrote: > Hi, Igniters! > > > Here are some news on our RPM packages initiative. > > 1. I’ve finished preliminary developing of Stage II version of RPM > packages [1]. Main “new feature” is — split design. Also I’ve added > package.sh script for automating package building process which will help > organise corresponding builds in TC as well as simplify process for > developers who wishes to have custom packages. > PR#3703 [2] is ready for review. Denis, in order to catch up with Apache > Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. > 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] > repositories on Apache Bintray. Though they are already prepared for > hosting RPM and DEB packages respectively, and there is a way of linking > them to apache.org/dist/ignite page, there is possible alternative in > storing there only plain directory layout corresponding to each repository > type (RPM and DEB) and manage this layout (repodata, distributions, > versions, etc.) by ourselves, having more control over repositories but > lacking some simplicity of deploying new releases. WDYT? Should we try > Cassandra approach? They are storing their DEB packages as I described > above [5]. > > Also — a question arose while I was working on this issue: which OSes (and > which versions of each) are we going to support (if we are going) in terms > of step-by-step list? Currently RPM packages are tested only with latest > CentOS (and, respectively — RHEL), but there are a lot more RPM-based > distributives [6] some of which are more o less popular among OS community > (ALT, Fedora, openSUSE, etc.). > > > [1] https://issues.apache.org/jira/browse/IGNITE-7647 > [2] https://github.com/apache/ignite/pull/3703 > [3] https://bintray.com/apache/ignite-rpm > [4] https://bintray.com/apache/ignite-deb > [5] https://bintray.com/apache/cassandra/debian#files/ > [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions > > > > > > On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: > > > > I suppose that most everything if not all from libs/options will go to > OPTIONAL (I’d call it simply ‘apache-ignite-libs'). > > More precise lib selection (if something from optional would better to > have in core package) will be discussed right after preliminary split > architecture agreement. > > > > > > > >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> wrote: > >> > >> I like idea of keeping simple system of modules, so +1 from me. > >> > >> Where optional libs (e.g Direct IO plugin) would be included, would it > be > >> core or optional? > >> > >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: > >> > >>>> > >>>>> > >>>>> How big would be a final core module? > >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s > own > >>>> package. > >>> > >>> > >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and > >>> propose the simplest modular system: > >>> > >>> - core module that includes basic Ignite capabilities including SQL, > >>> compute grid, service grid, k/v > >>> - optional module hosts the rest - ML, streamers integration (kafka, > >>> flink), kubernetes, etc. > >>> > >>> What do you think? > >>> > >>> -- > >>> Denis > >>> > >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> > wrote: > >>> > >>>> *DEB package > >>>> > >>>> > >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: > >>>>> > >>>>> Considering that DEV package for now is almost platform independent > >>> (its > >>>> a java application more or less), that package will work almost on any > >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. > >>>>> The only restriction is existence of systemctl (systemd) service > >>> manager > >>>> — we are dependent on it. > >>>>> > >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ > and > >>>> package has no arch or dist suffix — it will work on CentOS, RHEL, > >>> Fedora, > >>>> etc. with presence of aforementioned systemd. > >>>>> > >>>>> > >>>>> > >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> > >>>> wrote: > >>>>>> > >>>>>> Will Debian package work for Ubuntu? > >>>>>> > >>>>>> D. > >>>>>> > >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> > >>>> wrote: > >>>>>> > >>>>>>> Not a problem, rather nuisance. Also, when we will move to official > >>>>>>> repositories, there can be a problem from OS community. > >>>>>>> > >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package > >>>> build > >>>>>>> (package layout / install scripts) for speeding up things and > >>> excluding > >>>>>>> possible duplication and desynchronisation, so its a matter of ’sit > >>>> and do’ > >>>>>>> rather then some technical research. Thats why I rose discussion > >>> about > >>>>>>> future package architecture, so that after agreement I'm be able to > >>>> pack > >>>>>>> both RPM and DEB identically. > >>>>>>> > >>>>>>> Yet, if you insist, I can create DEB package according to current > RPM > >>>>>>> layout in no time. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < > [hidden email]> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Peter, > >>>>>>>> > >>>>>>>> I don't think the package size of 280M is going to be a problem at > >>>> all, > >>>>>>> but > >>>>>>>> what you are suggesting can be an improvement down the road. > >>>>>>>> > >>>>>>>> In the mean time, I think our top priority should be to provide > >>>> packages > >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. > >>>>>>>> > >>>>>>>> Agree? > >>>>>>>> > >>>>>>>> D. > >>>>>>>> > >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> > >>> wrote: > >>>>>>>> > >>>>>>>>> Hi, Igniters! > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd > >>> like > >>>> to > >>>>>>>>> move > >>>>>>>>> forward to further improve and widen AI delivery through > packages. > >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about > 280M+ > >>>> and, > >>>>>>> to > >>>>>>>>> improve usability and significantly reduce required download > >>> sizes, I > >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as > >>>> follows: > >>>>>>>>> - CORE > >>>>>>>>> - bin > >>>>>>>>> - config > >>>>>>>>> - libs (!optional) > >>>>>>>>> - OPTIONAL LIBS > >>>>>>>>> - BENCHMARKS > >>>>>>>>> - DOCS (?) > >>>>>>>>> - EXAMPLES > >>>>>>>>> - .NET PLATFORM FILES > >>>>>>>>> - C++ PLATFORM FILES > >>>>>>>>> > >>>>>>>>> This architecture, as I assume, will add flexibility (no reason > to > >>>>>>> download > >>>>>>>>> all 280M+ of binaries where you are to run only core node > >>>> functionality) > >>>>>>>>> and > >>>>>>>>> maintainability (you are in full control of what is installed on > >>> your > >>>>>>>>> system). > >>>>>>>>> > >>>>>>>>> After successful architecture choice, same scheme are planned to > be > >>>>>>> used in > >>>>>>>>> DEB packages as well. > >>>>>>>>> > >>>>>>>>> WDYT? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble. > com/ > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> > >>> > > > > |
|
|
No, not yet.
Currently we are discussing RPM packages only. I want to get all feedback and possible errors working on RPM packages, so that when we have stable agreed architecture and etc. I can recreate it in DEB packages without necessity to fix bugs in both RPM and DEB packages simultaneously. > On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <[hidden email]> wrote: > > Petr, > > I am confused. Do we already have Debian packages? > > D. > > On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <[hidden email]> wrote: > >> Hi, Igniters! >> >> >> Here are some news on our RPM packages initiative. >> >> 1. I’ve finished preliminary developing of Stage II version of RPM >> packages [1]. Main “new feature” is — split design. Also I’ve added >> package.sh script for automating package building process which will help >> organise corresponding builds in TC as well as simplify process for >> developers who wishes to have custom packages. >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. >> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] >> repositories on Apache Bintray. Though they are already prepared for >> hosting RPM and DEB packages respectively, and there is a way of linking >> them to apache.org/dist/ignite page, there is possible alternative in >> storing there only plain directory layout corresponding to each repository >> type (RPM and DEB) and manage this layout (repodata, distributions, >> versions, etc.) by ourselves, having more control over repositories but >> lacking some simplicity of deploying new releases. WDYT? Should we try >> Cassandra approach? They are storing their DEB packages as I described >> above [5]. >> >> Also — a question arose while I was working on this issue: which OSes (and >> which versions of each) are we going to support (if we are going) in terms >> of step-by-step list? Currently RPM packages are tested only with latest >> CentOS (and, respectively — RHEL), but there are a lot more RPM-based >> distributives [6] some of which are more o less popular among OS community >> (ALT, Fedora, openSUSE, etc.). >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-7647 >> [2] https://github.com/apache/ignite/pull/3703 >> [3] https://bintray.com/apache/ignite-rpm >> [4] https://bintray.com/apache/ignite-deb >> [5] https://bintray.com/apache/cassandra/debian#files/ >> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions >> >> >> >> >>> On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: >>> >>> I suppose that most everything if not all from libs/options will go to >> OPTIONAL (I’d call it simply ‘apache-ignite-libs'). >>> More precise lib selection (if something from optional would better to >> have in core package) will be discussed right after preliminary split >> architecture agreement. >>> >>> >>> >>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> wrote: >>>> >>>> I like idea of keeping simple system of modules, so +1 from me. >>>> >>>> Where optional libs (e.g Direct IO plugin) would be included, would it >> be >>>> core or optional? >>>> >>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: >>>> >>>>>> >>>>>>> >>>>>>> How big would be a final core module? >>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s >> own >>>>>> package. >>>>> >>>>> >>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and >>>>> propose the simplest modular system: >>>>> >>>>> - core module that includes basic Ignite capabilities including SQL, >>>>> compute grid, service grid, k/v >>>>> - optional module hosts the rest - ML, streamers integration (kafka, >>>>> flink), kubernetes, etc. >>>>> >>>>> What do you think? >>>>> >>>>> -- >>>>> Denis >>>>> >>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> >> wrote: >>>>> >>>>>> *DEB package >>>>>> >>>>>> >>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: >>>>>>> >>>>>>> Considering that DEV package for now is almost platform independent >>>>> (its >>>>>> a java application more or less), that package will work almost on any >>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >>>>>>> The only restriction is existence of systemctl (systemd) service >>>>> manager >>>>>> — we are dependent on it. >>>>>>> >>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ >> and >>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL, >>>>> Fedora, >>>>>> etc. with presence of aforementioned systemd. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email]> >>>>>> wrote: >>>>>>>> >>>>>>>> Will Debian package work for Ubuntu? >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> >>>>>> wrote: >>>>>>>> >>>>>>>>> Not a problem, rather nuisance. Also, when we will move to official >>>>>>>>> repositories, there can be a problem from OS community. >>>>>>>>> >>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package >>>>>> build >>>>>>>>> (package layout / install scripts) for speeding up things and >>>>> excluding >>>>>>>>> possible duplication and desynchronisation, so its a matter of ’sit >>>>>> and do’ >>>>>>>>> rather then some technical research. Thats why I rose discussion >>>>> about >>>>>>>>> future package architecture, so that after agreement I'm be able to >>>>>> pack >>>>>>>>> both RPM and DEB identically. >>>>>>>>> >>>>>>>>> Yet, if you insist, I can create DEB package according to current >> RPM >>>>>>>>> layout in no time. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < >> [hidden email]> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Peter, >>>>>>>>>> >>>>>>>>>> I don't think the package size of 280M is going to be a problem at >>>>>> all, >>>>>>>>> but >>>>>>>>>> what you are suggesting can be an improvement down the road. >>>>>>>>>> >>>>>>>>>> In the mean time, I think our top priority should be to provide >>>>>> packages >>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>>>>>>>>> >>>>>>>>>> Agree? >>>>>>>>>> >>>>>>>>>> D. >>>>>>>>>> >>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> >>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, Igniters! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >>>>> like >>>>>> to >>>>>>>>>>> move >>>>>>>>>>> forward to further improve and widen AI delivery through >> packages. >>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about >> 280M+ >>>>>> and, >>>>>>>>> to >>>>>>>>>>> improve usability and significantly reduce required download >>>>> sizes, I >>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >>>>>> follows: >>>>>>>>>>> - CORE >>>>>>>>>>> - bin >>>>>>>>>>> - config >>>>>>>>>>> - libs (!optional) >>>>>>>>>>> - OPTIONAL LIBS >>>>>>>>>>> - BENCHMARKS >>>>>>>>>>> - DOCS (?) >>>>>>>>>>> - EXAMPLES >>>>>>>>>>> - .NET PLATFORM FILES >>>>>>>>>>> - C++ PLATFORM FILES >>>>>>>>>>> >>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason >> to >>>>>>>>> download >>>>>>>>>>> all 280M+ of binaries where you are to run only core node >>>>>> functionality) >>>>>>>>>>> and >>>>>>>>>>> maintainability (you are in full control of what is installed on >>>>> your >>>>>>>>>>> system). >>>>>>>>>>> >>>>>>>>>>> After successful architecture choice, same scheme are planned to >> be >>>>>>>>> used in >>>>>>>>>>> DEB packages as well. >>>>>>>>>>> >>>>>>>>>>> WDYT? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble. >> com/ >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>> >> >> |
Re: Apache Ignite RPM packaging (Stage II)
|
|
Thanks, Petr!
I would love to test the package installation, but I can only do it on Ubuntu. Do you know when will we be able to get the Debian instructions, similar to this: https://ignite.apache.org/download.cgi#rpm-package D. On Wed, Mar 28, 2018 at 12:01 AM, Petr Ivanov <[hidden email]> wrote: > No, not yet. > > > Currently we are discussing RPM packages only. > I want to get all feedback and possible errors working on RPM packages, so > that when we have stable agreed architecture and etc. I can recreate it in > DEB packages without necessity to fix bugs in both RPM and DEB packages > simultaneously. > > > > > On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <[hidden email]> > wrote: > > > > Petr, > > > > I am confused. Do we already have Debian packages? > > > > D. > > > > On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <[hidden email]> > wrote: > > > >> Hi, Igniters! > >> > >> > >> Here are some news on our RPM packages initiative. > >> > >> 1. I’ve finished preliminary developing of Stage II version of RPM > >> packages [1]. Main “new feature” is — split design. Also I’ve added > >> package.sh script for automating package building process which will > help > >> organise corresponding builds in TC as well as simplify process for > >> developers who wishes to have custom packages. > >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache > >> Ignite 2.5 release, I’d greatly appreciate your help in finding > reviewer. > >> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] > >> repositories on Apache Bintray. Though they are already prepared for > >> hosting RPM and DEB packages respectively, and there is a way of linking > >> them to apache.org/dist/ignite page, there is possible alternative in > >> storing there only plain directory layout corresponding to each > repository > >> type (RPM and DEB) and manage this layout (repodata, distributions, > >> versions, etc.) by ourselves, having more control over repositories but > >> lacking some simplicity of deploying new releases. WDYT? Should we try > >> Cassandra approach? They are storing their DEB packages as I described > >> above [5]. > >> > >> Also — a question arose while I was working on this issue: which OSes > (and > >> which versions of each) are we going to support (if we are going) in > terms > >> of step-by-step list? Currently RPM packages are tested only with latest > >> CentOS (and, respectively — RHEL), but there are a lot more RPM-based > >> distributives [6] some of which are more o less popular among OS > community > >> (ALT, Fedora, openSUSE, etc.). > >> > >> > >> [1] https://issues.apache.org/jira/browse/IGNITE-7647 > >> [2] https://github.com/apache/ignite/pull/3703 > >> [3] https://bintray.com/apache/ignite-rpm > >> [4] https://bintray.com/apache/ignite-deb > >> [5] https://bintray.com/apache/cassandra/debian#files/ > >> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_ > distributions > >> > >> > >> > >> > >>> On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: > >>> > >>> I suppose that most everything if not all from libs/options will go to > >> OPTIONAL (I’d call it simply ‘apache-ignite-libs'). > >>> More precise lib selection (if something from optional would better to > >> have in core package) will be discussed right after preliminary split > >> architecture agreement. > >>> > >>> > >>> > >>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> > wrote: > >>>> > >>>> I like idea of keeping simple system of modules, so +1 from me. > >>>> > >>>> Where optional libs (e.g Direct IO plugin) would be included, would it > >> be > >>>> core or optional? > >>>> > >>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: > >>>> > >>>>>> > >>>>>>> > >>>>>>> How big would be a final core module? > >>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create > it’s > >> own > >>>>>> package. > >>>>> > >>>>> > >>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr > and > >>>>> propose the simplest modular system: > >>>>> > >>>>> - core module that includes basic Ignite capabilities including SQL, > >>>>> compute grid, service grid, k/v > >>>>> - optional module hosts the rest - ML, streamers integration (kafka, > >>>>> flink), kubernetes, etc. > >>>>> > >>>>> What do you think? > >>>>> > >>>>> -- > >>>>> Denis > >>>>> > >>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> > >> wrote: > >>>>> > >>>>>> *DEB package > >>>>>> > >>>>>> > >>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: > >>>>>>> > >>>>>>> Considering that DEV package for now is almost platform independent > >>>>> (its > >>>>>> a java application more or less), that package will work almost on > any > >>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. > >>>>>>> The only restriction is existence of systemctl (systemd) service > >>>>> manager > >>>>>> — we are dependent on it. > >>>>>>> > >>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ > >> and > >>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL, > >>>>> Fedora, > >>>>>> etc. with presence of aforementioned systemd. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan < > [hidden email]> > >>>>>> wrote: > >>>>>>>> > >>>>>>>> Will Debian package work for Ubuntu? > >>>>>>>> > >>>>>>>> D. > >>>>>>>> > >>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email] > > > >>>>>> wrote: > >>>>>>>> > >>>>>>>>> Not a problem, rather nuisance. Also, when we will move to > official > >>>>>>>>> repositories, there can be a problem from OS community. > >>>>>>>>> > >>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB > package > >>>>>> build > >>>>>>>>> (package layout / install scripts) for speeding up things and > >>>>> excluding > >>>>>>>>> possible duplication and desynchronisation, so its a matter of > ’sit > >>>>>> and do’ > >>>>>>>>> rather then some technical research. Thats why I rose discussion > >>>>> about > >>>>>>>>> future package architecture, so that after agreement I'm be able > to > >>>>>> pack > >>>>>>>>> both RPM and DEB identically. > >>>>>>>>> > >>>>>>>>> Yet, if you insist, I can create DEB package according to current > >> RPM > >>>>>>>>> layout in no time. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < > >> [hidden email]> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Peter, > >>>>>>>>>> > >>>>>>>>>> I don't think the package size of 280M is going to be a problem > at > >>>>>> all, > >>>>>>>>> but > >>>>>>>>>> what you are suggesting can be an improvement down the road. > >>>>>>>>>> > >>>>>>>>>> In the mean time, I think our top priority should be to provide > >>>>>> packages > >>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. > >>>>>>>>>> > >>>>>>>>>> Agree? > >>>>>>>>>> > >>>>>>>>>> D. > >>>>>>>>>> > >>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> > >>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi, Igniters! > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd > >>>>> like > >>>>>> to > >>>>>>>>>>> move > >>>>>>>>>>> forward to further improve and widen AI delivery through > >> packages. > >>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about > >> 280M+ > >>>>>> and, > >>>>>>>>> to > >>>>>>>>>>> improve usability and significantly reduce required download > >>>>> sizes, I > >>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as > >>>>>> follows: > >>>>>>>>>>> - CORE > >>>>>>>>>>> - bin > >>>>>>>>>>> - config > >>>>>>>>>>> - libs (!optional) > >>>>>>>>>>> - OPTIONAL LIBS > >>>>>>>>>>> - BENCHMARKS > >>>>>>>>>>> - DOCS (?) > >>>>>>>>>>> - EXAMPLES > >>>>>>>>>>> - .NET PLATFORM FILES > >>>>>>>>>>> - C++ PLATFORM FILES > >>>>>>>>>>> > >>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason > >> to > >>>>>>>>> download > >>>>>>>>>>> all 280M+ of binaries where you are to run only core node > >>>>>> functionality) > >>>>>>>>>>> and > >>>>>>>>>>> maintainability (you are in full control of what is installed > on > >>>>> your > >>>>>>>>>>> system). > >>>>>>>>>>> > >>>>>>>>>>> After successful architecture choice, same scheme are planned > to > >> be > >>>>>>>>> used in > >>>>>>>>>>> DEB packages as well. > >>>>>>>>>>> > >>>>>>>>>>> WDYT? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble. > >> com/ > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>> > >>> > >> > >> > > |
|
|
I can start preparing DEB packages right after adding RPM build to nightly release build (as an experiment / experience for future addition of packages build into release process) basing on current RPM architecture.
I will create branch from IGNITE-7647, then. > On 28 Mar 2018, at 10:06, Dmitriy Setrakyan <[hidden email]> wrote: > > Thanks, Petr! > > I would love to test the package installation, but I can only do it on > Ubuntu. Do you know when will we be able to get the Debian instructions, > similar to this: > > https://ignite.apache.org/download.cgi#rpm-package > > D. > > On Wed, Mar 28, 2018 at 12:01 AM, Petr Ivanov <[hidden email]> wrote: > >> No, not yet. >> >> >> Currently we are discussing RPM packages only. >> I want to get all feedback and possible errors working on RPM packages, so >> that when we have stable agreed architecture and etc. I can recreate it in >> DEB packages without necessity to fix bugs in both RPM and DEB packages >> simultaneously. >> >> >> >>> On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <[hidden email]> >> wrote: >>> >>> Petr, >>> >>> I am confused. Do we already have Debian packages? >>> >>> D. >>> >>> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <[hidden email]> >> wrote: >>> >>>> Hi, Igniters! >>>> >>>> >>>> Here are some news on our RPM packages initiative. >>>> >>>> 1. I’ve finished preliminary developing of Stage II version of RPM >>>> packages [1]. Main “new feature” is — split design. Also I’ve added >>>> package.sh script for automating package building process which will >> help >>>> organise corresponding builds in TC as well as simplify process for >>>> developers who wishes to have custom packages. >>>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >>>> Ignite 2.5 release, I’d greatly appreciate your help in finding >> reviewer. >>>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] >>>> repositories on Apache Bintray. Though they are already prepared for >>>> hosting RPM and DEB packages respectively, and there is a way of linking >>>> them to apache.org/dist/ignite page, there is possible alternative in >>>> storing there only plain directory layout corresponding to each >> repository >>>> type (RPM and DEB) and manage this layout (repodata, distributions, >>>> versions, etc.) by ourselves, having more control over repositories but >>>> lacking some simplicity of deploying new releases. WDYT? Should we try >>>> Cassandra approach? They are storing their DEB packages as I described >>>> above [5]. >>>> >>>> Also — a question arose while I was working on this issue: which OSes >> (and >>>> which versions of each) are we going to support (if we are going) in >> terms >>>> of step-by-step list? Currently RPM packages are tested only with latest >>>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based >>>> distributives [6] some of which are more o less popular among OS >> community >>>> (ALT, Fedora, openSUSE, etc.). >>>> >>>> >>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647 >>>> [2] https://github.com/apache/ignite/pull/3703 >>>> [3] https://bintray.com/apache/ignite-rpm >>>> [4] https://bintray.com/apache/ignite-deb >>>> [5] https://bintray.com/apache/cassandra/debian#files/ >>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_ >> distributions >>>> >>>> >>>> >>>> >>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: >>>>> >>>>> I suppose that most everything if not all from libs/options will go to >>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs'). >>>>> More precise lib selection (if something from optional would better to >>>> have in core package) will be discussed right after preliminary split >>>> architecture agreement. >>>>> >>>>> >>>>> >>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> >> wrote: >>>>>> >>>>>> I like idea of keeping simple system of modules, so +1 from me. >>>>>> >>>>>> Where optional libs (e.g Direct IO plugin) would be included, would it >>>> be >>>>>> core or optional? >>>>>> >>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: >>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> How big would be a final core module? >>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create >> it’s >>>> own >>>>>>>> package. >>>>>>> >>>>>>> >>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr >> and >>>>>>> propose the simplest modular system: >>>>>>> >>>>>>> - core module that includes basic Ignite capabilities including SQL, >>>>>>> compute grid, service grid, k/v >>>>>>> - optional module hosts the rest - ML, streamers integration (kafka, >>>>>>> flink), kubernetes, etc. >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> -- >>>>>>> Denis >>>>>>> >>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> >>>> wrote: >>>>>>> >>>>>>>> *DEB package >>>>>>>> >>>>>>>> >>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: >>>>>>>>> >>>>>>>>> Considering that DEV package for now is almost platform independent >>>>>>> (its >>>>>>>> a java application more or less), that package will work almost on >> any >>>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >>>>>>>>> The only restriction is existence of systemctl (systemd) service >>>>>>> manager >>>>>>>> — we are dependent on it. >>>>>>>>> >>>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ >>>> and >>>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL, >>>>>>> Fedora, >>>>>>>> etc. with presence of aforementioned systemd. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan < >> [hidden email]> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Will Debian package work for Ubuntu? >>>>>>>>>> >>>>>>>>>> D. >>>>>>>>>> >>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email] >>> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to >> official >>>>>>>>>>> repositories, there can be a problem from OS community. >>>>>>>>>>> >>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB >> package >>>>>>>> build >>>>>>>>>>> (package layout / install scripts) for speeding up things and >>>>>>> excluding >>>>>>>>>>> possible duplication and desynchronisation, so its a matter of >> ’sit >>>>>>>> and do’ >>>>>>>>>>> rather then some technical research. Thats why I rose discussion >>>>>>> about >>>>>>>>>>> future package architecture, so that after agreement I'm be able >> to >>>>>>>> pack >>>>>>>>>>> both RPM and DEB identically. >>>>>>>>>>> >>>>>>>>>>> Yet, if you insist, I can create DEB package according to current >>>> RPM >>>>>>>>>>> layout in no time. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < >>>> [hidden email]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Peter, >>>>>>>>>>>> >>>>>>>>>>>> I don't think the package size of 280M is going to be a problem >> at >>>>>>>> all, >>>>>>>>>>> but >>>>>>>>>>>> what you are suggesting can be an improvement down the road. >>>>>>>>>>>> >>>>>>>>>>>> In the mean time, I think our top priority should be to provide >>>>>>>> packages >>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >>>>>>>>>>>> >>>>>>>>>>>> Agree? >>>>>>>>>>>> >>>>>>>>>>>> D. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> >>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, Igniters! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >>>>>>> like >>>>>>>> to >>>>>>>>>>>>> move >>>>>>>>>>>>> forward to further improve and widen AI delivery through >>>> packages. >>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about >>>> 280M+ >>>>>>>> and, >>>>>>>>>>> to >>>>>>>>>>>>> improve usability and significantly reduce required download >>>>>>> sizes, I >>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >>>>>>>> follows: >>>>>>>>>>>>> - CORE >>>>>>>>>>>>> - bin >>>>>>>>>>>>> - config >>>>>>>>>>>>> - libs (!optional) >>>>>>>>>>>>> - OPTIONAL LIBS >>>>>>>>>>>>> - BENCHMARKS >>>>>>>>>>>>> - DOCS (?) >>>>>>>>>>>>> - EXAMPLES >>>>>>>>>>>>> - .NET PLATFORM FILES >>>>>>>>>>>>> - C++ PLATFORM FILES >>>>>>>>>>>>> >>>>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason >>>> to >>>>>>>>>>> download >>>>>>>>>>>>> all 280M+ of binaries where you are to run only core node >>>>>>>> functionality) >>>>>>>>>>>>> and >>>>>>>>>>>>> maintainability (you are in full control of what is installed >> on >>>>>>> your >>>>>>>>>>>>> system). >>>>>>>>>>>>> >>>>>>>>>>>>> After successful architecture choice, same scheme are planned >> to >>>> be >>>>>>>>>>> used in >>>>>>>>>>>>> DEB packages as well. >>>>>>>>>>>>> >>>>>>>>>>>>> WDYT? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble. >>>> com/ >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>>> >> >> |
|
|
In reply to this post by vveider
>
> We can either > store files as is (as Cassandra does, see link below) in something > that is called Generic Repository — this way we manage directory layouts > ourselves and have more control over what is published and how > or > store files in prepared RPM / DEB repositories where we only upload > packages and everything else repository does itself — this way we rely on > Bintray’s implementations of corresponding repositories and have lower > control over directory layout, but the overall process seems to be simpler. Petr, he I would lean towards your experience and opinion. Personally, I can't see any significant advantages or disadvantages of either approach. This suggests me that we should go for the simplest (2nd) way. -- Denis On Tue, Mar 27, 2018 at 11:59 PM, Petr Ivanov <[hidden email]> wrote: > > > On 28 Mar 2018, at 00:18, Denis Magda <[hidden email]> wrote: > > Petr, thanks for the update. > > >> 1. I’ve finished preliminary developing of Stage II version of RPM >> packages [1]. Main “new feature” is — split design. Also I’ve added >> package.sh script for automating package building process which will help >> organise corresponding builds in TC as well as simplify process for >> developers who wishes to have custom packages. >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. > > > *Anton*, would you be able to check up Petr's pull-request? > > >> corresponding to each repository type (RPM and DEB) and manage this >> layout (repodata, distributions, versions, etc.) by ourselves, having more >> control over repositories but lacking some simplicity of deploying new >> releases. WDYT? Should we try Cassandra approach? They are storing their >> DEB packages as I described above [5]. > > > Petr, do bintray repos come for free for ASF project? Are there any > limitations in regards total repository or package size? > > > As I see, ASF has premium free account at Bintray with, I guess, virutally > limitless posiiblilities (the only I stumbled across is limit to uploading > files more that 250Mb through UI — upload is possible through API). Yet, > I’ll ask INFRA specifically about this. > > > > WDYT? Should we try Cassandra approach? They are storing their DEB >> packages as I described above [5]. > > > I'm not sure I got the question. Could you show examples of both > approaches? > > > We can either > store files as is (as Cassandra does, see link below) in something > that is called Generic Repository — this way we manage directory layouts > ourselves and have more control over what is published and how > or > store files in prepared RPM / DEB repositories where we only upload > packages and everything else repository does itself — this way we rely on > Bintray’s implementations of corresponding repositories and have lower > control over directory layout, but the overall process seems to be simpler. > > > > Also — a question arose while I was working on this issue: which OSes (and >> which versions of each) are we going to support (if we are going) in terms >> of step-by-step list? Currently RPM packages are tested only with latest >> CentOS (and, respectively — RHEL), but there are a lot more RPM-based >> distributives [6] some of which are more o less popular among OS community >> (ALT, Fedora, openSUSE, etc.). > > > I would test RHEL and Ubuntu for sure, of the last versions. As for the > other distributions, I'd like to hear community opinion. > > > There is minor to none necessity of testing packages on RHEL — CentOS is > binary clone. > Ubuntu is out of scope because it uses DEB packages and currently we are > discussing RPM only. > > > > -- > Denis > > > On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <[hidden email]> wrote: > >> Hi, Igniters! >> >> >> Here are some news on our RPM packages initiative. >> >> 1. I’ve finished preliminary developing of Stage II version of RPM >> packages [1]. Main “new feature” is — split design. Also I’ve added >> package.sh script for automating package building process which will help >> organise corresponding builds in TC as well as simplify process for >> developers who wishes to have custom packages. >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. >> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] >> repositories on Apache Bintray. Though they are already prepared for >> hosting RPM and DEB packages respectively, and there is a way of linking >> them to apache.org/dist/ignite page, there is possible alternative in >> storing there only plain directory layout corresponding to each repository >> type (RPM and DEB) and manage this layout (repodata, distributions, >> versions, etc.) by ourselves, having more control over repositories but >> lacking some simplicity of deploying new releases. WDYT? Should we try >> Cassandra approach? They are storing their DEB packages as I described >> above [5]. >> >> Also — a question arose while I was working on this issue: which OSes >> (and which versions of each) are we going to support (if we are going) in >> terms of step-by-step list? Currently RPM packages are tested only with >> latest CentOS (and, respectively — RHEL), but there are a lot more >> RPM-based distributives [6] some of which are more o less popular among OS >> community (ALT, Fedora, openSUSE, etc.). >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-7647 >> [2] https://github.com/apache/ignite/pull/3703 >> [3] https://bintray.com/apache/ignite-rpm >> [4] https://bintray.com/apache/ignite-deb >> [5] https://bintray.com/apache/cassandra/debian#files/ >> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions >> >> >> >> >> > On 15 Mar 2018, at 22:15, Petr Ivanov <[hidden email]> wrote: >> > >> > I suppose that most everything if not all from libs/options will go to >> OPTIONAL (I’d call it simply ‘apache-ignite-libs'). >> > More precise lib selection (if something from optional would better to >> have in core package) will be discussed right after preliminary split >> architecture agreement. >> > >> > >> > >> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <[hidden email]> wrote: >> >> >> >> I like idea of keeping simple system of modules, so +1 from me. >> >> >> >> Where optional libs (e.g Direct IO plugin) would be included, would it >> be >> >> core or optional? >> >> >> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <[hidden email]>: >> >> >> >>>> >> >>>>> >> >>>>> How big would be a final core module? >> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create >> it’s own >> >>>> package. >> >>> >> >>> >> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and >> >>> propose the simplest modular system: >> >>> >> >>> - core module that includes basic Ignite capabilities including SQL, >> >>> compute grid, service grid, k/v >> >>> - optional module hosts the rest - ML, streamers integration (kafka, >> >>> flink), kubernetes, etc. >> >>> >> >>> What do you think? >> >>> >> >>> -- >> >>> Denis >> >>> >> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <[hidden email]> >> wrote: >> >>> >> >>>> *DEB package >> >>>> >> >>>> >> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <[hidden email]> wrote: >> >>>>> >> >>>>> Considering that DEV package for now is almost platform independent >> >>> (its >> >>>> a java application more or less), that package will work almost on >> any >> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >> >>>>> The only restriction is existence of systemctl (systemd) service >> >>> manager >> >>>> — we are dependent on it. >> >>>>> >> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ >> and >> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL, >> >>> Fedora, >> >>>> etc. with presence of aforementioned systemd. >> >>>>> >> >>>>> >> >>>>> >> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <[hidden email] >> > >> >>>> wrote: >> >>>>>> >> >>>>>> Will Debian package work for Ubuntu? >> >>>>>> >> >>>>>> D. >> >>>>>> >> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <[hidden email]> >> >>>> wrote: >> >>>>>> >> >>>>>>> Not a problem, rather nuisance. Also, when we will move to >> official >> >>>>>>> repositories, there can be a problem from OS community. >> >>>>>>> >> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB >> package >> >>>> build >> >>>>>>> (package layout / install scripts) for speeding up things and >> >>> excluding >> >>>>>>> possible duplication and desynchronisation, so its a matter of >> ’sit >> >>>> and do’ >> >>>>>>> rather then some technical research. Thats why I rose discussion >> >>> about >> >>>>>>> future package architecture, so that after agreement I'm be able >> to >> >>>> pack >> >>>>>>> both RPM and DEB identically. >> >>>>>>> >> >>>>>>> Yet, if you insist, I can create DEB package according to current >> RPM >> >>>>>>> layout in no time. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < >> [hidden email]> >> >>>>>>> wrote: >> >>>>>>>> >> >>>>>>>> Peter, >> >>>>>>>> >> >>>>>>>> I don't think the package size of 280M is going to be a problem >> at >> >>>> all, >> >>>>>>> but >> >>>>>>>> what you are suggesting can be an improvement down the road. >> >>>>>>>> >> >>>>>>>> In the mean time, I think our top priority should be to provide >> >>>> packages >> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >> >>>>>>>> >> >>>>>>>> Agree? >> >>>>>>>> >> >>>>>>>> D. >> >>>>>>>> >> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <[hidden email]> >> >>> wrote: >> >>>>>>>> >> >>>>>>>>> Hi, Igniters! >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >> >>> like >> >>>> to >> >>>>>>>>> move >> >>>>>>>>> forward to further improve and widen AI delivery through >> packages. >> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about >> 280M+ >> >>>> and, >> >>>>>>> to >> >>>>>>>>> improve usability and significantly reduce required download >> >>> sizes, I >> >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >> >>>> follows: >> >>>>>>>>> - CORE >> >>>>>>>>> - bin >> >>>>>>>>> - config >> >>>>>>>>> - libs (!optional) >> >>>>>>>>> - OPTIONAL LIBS >> >>>>>>>>> - BENCHMARKS >> >>>>>>>>> - DOCS (?) >> >>>>>>>>> - EXAMPLES >> >>>>>>>>> - .NET PLATFORM FILES >> >>>>>>>>> - C++ PLATFORM FILES >> >>>>>>>>> >> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason >> to >> >>>>>>> download >> >>>>>>>>> all 280M+ of binaries where you are to run only core node >> >>>> functionality) >> >>>>>>>>> and >> >>>>>>>>> maintainability (you are in full control of what is installed on >> >>> your >> >>>>>>>>> system). >> >>>>>>>>> >> >>>>>>>>> After successful architecture choice, same scheme are planned >> to be >> >>>>>>> used in >> >>>>>>>>> DEB packages as well. >> >>>>>>>>> >> >>>>>>>>> WDYT? >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> Sent from: http://apache-ignite-developer >> s.2346864.n4.nabble.com/ >> >>>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>>> >> >>> >> > >> >> > > |
Re: Apache Ignite RPM packaging (Stage II)
|
|
In reply to this post by vveider
Peter, great work!
I'll check up PR in the near time. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |