Skip to main content

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

  • From: Yiteng Zhang < >
  • To: Bart Smaalders < >, Erik Trauschke < >
  • Cc:
  • Subject: [pkg-discuss] Re: code review request: pkg dehydrate/rehydrate
  • Date: Fri, 13 Jun 2014 11:53:31 -0700

On 06/ 9/14 02:03 PM, Yiteng Zhang wrote:
On 06/ 5/14 01:18 PM, Bart Smaalders wrote:
On 05/29/14 16:23, Yiteng Zhang wrote:
Hi folks,


This looks like a good start to me.

How does it perform on an image containing solaris-large-server, both in
time taken and space saved?

Some areas of concern:

In __gen_hydrate_actions, you hard-code in exceptions for etc/zones/* files. This would be
much better done by adding 'dehydrate=false' to those files in their own manifests. That
way if we find other files that need to be retained, we can add the same attribute - or
other developers of those packages can. This would need to be done in ON -but it is
file-specific metadata, and belongs in the packages.

In __gen_hydrate_actions, you call get_manifest with ignore_excludes=True. Why?

It is possible to add a publisher and install packages from it _after_ dehydration.
I think this will break your use of 'all' in the dehydrated_pubs list; you're probably better
off not using 'all' but just dealing with the list of publishers.

Pkg operations performed after dehydration may malfunction in interesting ways.
Do we need to prevent this or test it? What happens if I add a package from
the solaris publisher after doing dehydration? What happens if I rehydrate after
doing that? It may just work, but we either need to test it or prevent users
from doing silly things.

I am going to implement the idea to prevent silly pkg operations like this:
If the pkg dehydrate command succeeds, a property named dehydrated with the
names of the publishers specified as the value is set on the image. I will change
the behavior of pkg install/uninstall/update/change-facet/change-variant to first check
if the pkg operations are going to modify the package content delivered by the dehydrated
publishers. If so, we will warn the users and do nothing. If not, we do what the pkg
operations were supposed to do.

For example, if users dehydrated on publisher 'solaris', then we will prevent users installing
any package from the 'solaris' publisher, but we won't prevent users installing packages
from other publishers.

At this point, I find it might be a little tricky to implement the idea because we couldn't
simply check the publishers at the very beginning but need to filter the fmri lists by the
publishers. But I believe we can reach there.

Any suggestion or objection?


Hi,

I did implementation of these two approaches, one is to prevent users doing pkg operations between
pkg dehydrate-rehydrate, the other is to allow users doing pkg operations between pkg dehydrate-rehydrate.
And I find it is cleaner and better to use the latter approach.

This is because:
1) If we prevent users doing pkg operations, we need to check if the pkg operation is going to modify
the pkg content under the dehydrated publishers, and it is kind of overwhelming to add this check for
every pkg operations. Also once we introduce new pkg command, we will need to add this check too.
I think it is better to get everything done inside pkg dehydrate/rehydrate themselves and other pkg
operations should not worry about if they will break pkg dehydrate/rehydrate.
2) I find a clean way to make pkg dehydrate/rehydrate work if users do pkg operations between them.
I don't even need to create a new property "dehydrated" for the image now.

So I decided to allow users doing pkg operations between pkg dehydrate-rehydrate and test it.

Also I created pkg fix's own progress tracker's methods. For now, I modify pkg verify's progress tracker
to print out the line "Verifying pkg_name", which seems not elegant. I think there is more work to do here:
pkg verify should also move into api and its progress tracker methodes should be updated. But it is out
of scope for this project. Probably let us do it later.

Other main changes for this webrev are:
Updated doc/client_api_versions.txt, made the version to be 81
Updated src/man/pkg.5, added a new attribute "dehydrate" to File Action

incremental: https://ips.java.net/webrev/yitezhan/pkg_hydrate_2ag1/
webrev: https://ips.java.net/webrev/yitezhan/pkg_hydrate_2/

The change for on12-gate is as follow:
$ hg diff
diff -r 633d4938f39a usr/src/pkg/manifests/system:zones.p5m
--- a/usr/src/pkg/manifests/system:zones.p5m Wed Jun 11 12:48:04 2014 -0700
+++ b/usr/src/pkg/manifests/system:zones.p5m Fri Jun 13 11:31:10 2014 -0700
@@ -42,7 +42,8 @@
 file path=etc/zones/SYSblank.xml mode=0444 variant.opensolaris.zone=global
 file path=etc/zones/SYSdefault-shared-ip.xml mode=0444 \
     variant.opensolaris.zone=global
-file path=etc/zones/SYSdefault.xml mode=0444 variant.opensolaris.zone=global
+file path=etc/zones/SYSdefault.xml mode=0444 variant.opensolaris.zone=global \
+    dehydrate=false
 link path=etc/zones/SYSsolaris.xml target=./SYSdefault.xml \
     variant.opensolaris.zone=global
file path=etc/zones/index group=sys original_name=SUNWzone:etc/zones/index \

They have already added a attribute "preserve=true" for the file etc/zones/index, which seems magical to me...
I think I don't have the right to push to on12-gate, do I?

Please let me know your opinion!
Thanks,
Yiteng




[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Bart Smaalders 06/05/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/05/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/09/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/13/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/19/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Danek Duvall 06/23/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/24/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Shawn Walker 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Yiteng Zhang 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Shawn Walker 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Danek Duvall 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Shawn Walker 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Danek Duvall 06/26/2014

[pkg-discuss] Re: code review request: pkg dehydrate/rehydrate

Shawn Walker 06/26/2014
 
 
Close
loading
Please Confirm
Close