Paste number 358124: Script to flag bad github packages

Index of paste annotations: 1

Paste number 358124: Script to flag bad github packages
Pasted by: Apteryx
When:4 years, 1 month ago
Share:Tweet this! | http://paste.lisp.org/+7OBW
Channel:None
Paste contents:
Raw Source | XML | Display As
;;; A script to find packages possibly affected by GitHub
;;; infrastructure update that caused minor changes in the
;;; automatically generated tarballs.

(use-modules (ice-9 match)
	     (ice-9 textual-ports)
	     (gnu packages)
	     (guix build utils)
	     (guix download)
	     (guix packages)
	     (srfi srfi-1))

(define (problematic-uri? uri)

  (define (contains-github-archive? uri)
    (regexp-match? (string-match "github.com/.*/archive/" uri)))

  ;; URI can be a string or a list of string.
  (match uri
    ((uri1 uri2 ...)			;match list of strings
     (not (null? (filter contains-github-archive? uri))))
    (uri1				;match string
     (contains-github-archive? uri1))))

(define (problematic-github-package? package)
  (let ((source (package-source package)))
    (and (origin? source)
	 (eq? (origin-method source) url-fetch)
	 (problematic-uri? (origin-uri source)))))

(define problematic-github-packages
  ;; List of all the potentially problematic GitHub packages.
  (fold-packages (lambda (p r)
		   (if (problematic-github-package? p)
		       (cons p r)
		       r))
		 '()))

(define homebrew-affected-packages
  ;; List of the Homebrew flagged packages that were found in Guix
  ;; potentially problematic packages.
  (with-input-from-file "/home/maxim/src/varia/known-affected-packages.txt"
    (lambda ()
      (let* ((content-as-string (get-string-all (current-input-port)))
	     (lines (string-split content-as-string #\newline)))

	(define (line->package-name line)
	  (first (string-split line #\:)))

	(define package-raw-names
	  (map line->package-name lines))

	(define (arobas->hyphen char)
	  (if (char=? char #\@)
	      #\-
	      char))

	(define package-names+versions
	  (map (lambda (package-raw-name)
		 (string-map arobas->hyphen package-raw-name))
	       package-raw-names))

	(define package-name-version-tuples
	  (map (compose list package-name->name+version)
	       package-names+versions))

	(define homebrew-packages-in-guix
	  (filter package?
		  (append-map (lambda (args)
				(apply find-best-packages-by-name args))
			      package-name-version-tuples)))

	;; Only keep homebrew affected packages found in our potentially
	;; problematic packages list.
	(lset-intersection eq?
			   homebrew-packages-in-guix
			   problematic-github-packages)))))

(define (fetch-package-source))
(define (main)
  "Find and print the names of the potentially problematic GitHub packages."
  (define (print-packages! packages)
    (for-each (lambda (name)
		(format #t "~a~%" name))
	      (sort (map package-name packages) string<?))
    (format #t "~%"))

  (format #t "Number of Homebrew flagged GitHub packages in Guix: ~a~%"
	  (length homebrew-affected-packages))
  (print-packages! homebrew-affected-packages)

  (format #t "Number of potentially problematic GitHub packages: ~a~%"
	  (length problematic-github-packages))
  (print-packages! problematic-github-packages))

;;; Run the program.
(main)

Annotations for this paste:

Annotation number 1: Homebrew data
Pasted by: Apteryx
When:4 years, 1 month ago
Share:Tweet this! | http://paste.lisp.org/+7OBW/1
Paste contents:
Raw Source | Display As
airspy: https://github.com/airspy/host/archive/v1.0.9.tar.gz
algernon: https://github.com/xyproto/algernon/archive/1.5.1.tar.gz
amazon-ecs-cli: https://github.com/aws/amazon-ecs-cli/archive/v0.6.4.tar.gz
apache-brooklyn-cli: https://github.com/apache/brooklyn-client/archive/rel/apache-brooklyn-0.11.0.tar.gz
aptly: https://github.com/smira/aptly/archive/v1.1.1.tar.gz
arabica: https://github.com/jezhiggins/arabica/archive/2016-January.tar.gz
assh: https://github.com/moul/advanced-ssh-config/archive/v2.6.0.tar.gz
aws-apigateway-importer: https://github.com/awslabs/aws-apigateway-importer/archive/aws-apigateway-importer-1.0.1.tar.gz
aws-sdk-cpp: https://github.com/aws/aws-sdk-cpp/archive/1.1.50.tar.gz
azure-cli: https://github.com/Azure/azure-xplat-cli/archive/v0.10.15-July2017.tar.gz
bartycrouch: https://github.com/Flinesoft/BartyCrouch/archive/3.8.1.tar.gz
bazel@0.2: https://github.com/bazelbuild/bazel/archive/0.2.3.tar.gz
blink1: https://github.com/todbot/blink1/archive/v1.98.tar.gz
bluepill: https://github.com/linkedin/bluepill/archive/v1.1.2.tar.gz
c14-cli: https://github.com/online-net/c14-cli/archive/0.1.tar.gz
certbot: https://github.com/certbot/certbot/archive/v0.18.1.tar.gz
cjdns: https://github.com/cjdelisle/cjdns/archive/cjdns-v20.tar.gz
cli53: https://github.com/barnybug/cli53/archive/0.8.9.tar.gz
cocoapods: https://github.com/CocoaPods/CocoaPods/archive/1.3.1.tar.gz
codequery: https://github.com/ruben2020/codequery/archive/v0.21.0.tar.gz
convox: https://github.com/convox/rack/archive/20170818204318.tar.gz
cookiecutter: https://github.com/audreyr/cookiecutter/archive/1.5.1.tar.gz
cpprestsdk: https://github.com/Microsoft/cpprestsdk/archive/v2.9.1.tar.gz
deisctl: https://github.com/deis/deis/archive/v1.13.4.tar.gz
docker-machine-completion: https://github.com/docker/machine/archive/v0.12.2.tar.gz
docker-machine-driver-vultr: https://github.com/janeczku/docker-machine-vultr/archive/v1.4.0.tar.gz
docker-machine-parallels: https://github.com/Parallels/docker-machine-parallels/archive/v1.2.3.tar.gz
echoprint-codegen: https://github.com/echonest/echoprint-codegen/archive/v4.12.tar.gz
erlang: https://github.com/erlang/otp/archive/OTP-20.0.4.tar.gz
erlang@17: https://github.com/erlang/otp/archive/OTP-17.5.6.9.tar.gz
erlang@18: https://github.com/erlang/otp/archive/OTP-18.3.4.tar.gz
erlang@19: https://github.com/erlang/otp/archive/OTP-19.3.tar.gz
flow: https://github.com/facebook/flow/archive/v0.54.1.tar.gz
gearboy: https://github.com/drhelius/Gearboy/archive/gearboy-2.3.1.tar.gz
gearsystem: https://github.com/drhelius/Gearsystem/archive/gearsystem-2.2.tar.gz
git-lfs: https://github.com/git-lfs/git-lfs/archive/v2.2.1.tar.gz
git-town: https://github.com/Originate/git-town/archive/v4.2.1.tar.gz
giter8: https://github.com/foundweekends/giter8/archive/v0.9.0.tar.gz
glbinding: https://github.com/cginternals/glbinding/archive/v2.1.3.tar.gz
globjects: https://github.com/cginternals/globjects/archive/v1.0.0.tar.gz
gollum: https://github.com/trivago/gollum/archive/v0.4.5.tar.gz
gomplate: https://github.com/hairyhenderson/gomplate/archive/v2.0.1.tar.gz
google-java-format: https://github.com/google/google-java-format/archive/google-java-format-1.4.tar.gz
gosu: https://github.com/gosu-lang/gosu-lang/archive/v1.14.6.tar.gz
hh: https://github.com/dvorka/hstr/archive/1.22.tar.gz
hub: https://github.com/github/hub/archive/v2.2.9.tar.gz
ice: https://github.com/zeroc-ice/ice/archive/v3.7.0.tar.gz
io: https://github.com/stevedekorte/io/archive/2017.09.06.tar.gz
jmxtrans: https://github.com/jmxtrans/jmxtrans/archive/jmxtrans-parent-267.tar.gz
kompose: https://github.com/kubernetes/kompose/archive/v1.1.0.tar.gz
kops: https://github.com/kubernetes/kops/archive/1.7.0.tar.gz
kube-aws: https://github.com/kubernetes-incubator/kube-aws/archive/v0.9.8.tar.gz
kubernetes-cli@1.3: https://github.com/kubernetes/kubernetes/archive/v1.3.10.tar.gz
libgit2: https://github.com/libgit2/libgit2/archive/v0.26.0.tar.gz
liquigraph: https://github.com/fbiville/liquigraph/archive/liquigraph-3.0.1.tar.gz
logtalk: https://github.com/LogtalkDotOrg/logtalk3/archive/lgt3112stable.tar.gz
mame: https://github.com/mamedev/mame/archive/mame0189.tar.gz
metashell: https://github.com/metashell/metashell/archive/v3.0.0.tar.gz
minimesos: https://github.com/ContainerSolutions/minimesos/archive/0.13.0.tar.gz
mogenerator: https://github.com/rentzsch/mogenerator/archive/1.31.tar.gz
monax: https://github.com/monax/monax/archive/v0.18.0.tar.gz
mongoose: https://github.com/cesanta/mongoose/archive/6.8.tar.gz
nats-streaming-server: https://github.com/nats-io/nats-streaming-server/archive/v0.5.0.tar.gz
ndpi: https://github.com/ntop/nDPI/archive/2.0.tar.gz
opencv: https://github.com/opencv/opencv/archive/3.3.0.tar.gz
openvdb: https://github.com/dreamworksanimation/openvdb/archive/v4.0.2.tar.gz
pcap_dnsproxy: https://github.com/chengr28/Pcap_DNSProxy/archive/v0.4.9.0.tar.gz
ponscripter-sekai: https://github.com/sekaiproject/ponscripter-fork/archive/v0.0.6.tar.gz
prometheus: https://github.com/prometheus/prometheus/archive/v1.7.1.tar.gz
protobuf: https://github.com/google/protobuf/archive/v3.4.0.tar.gz
protobuf@3.1: https://github.com/google/protobuf/archive/v3.1.0.tar.gz
pumba: https://github.com/gaia-adm/pumba/archive/0.4.5.tar.gz
rclone: https://github.com/ncw/rclone/archive/v1.37.tar.gz
redex: https://github.com/facebook/redex/archive/v1.1.0.tar.gz
robot-framework: https://github.com/robotframework/robotframework/archive/3.0.2.tar.gz
rom-tools: https://github.com/mamedev/mame/archive/mame0189.tar.gz
sops: https://github.com/mozilla/sops/archive/2.0.10.tar.gz
ssdb: https://github.com/ideawu/ssdb/archive/1.9.4.tar.gz
swagger-codegen: https://github.com/swagger-api/swagger-codegen/archive/v2.2.3.tar.gz
swift: https://github.com/apple/swift/archive/swift-3.1.1-RELEASE.tar.gz
swiftformat: https://github.com/nicklockwood/SwiftFormat/archive/0.29.5.tar.gz
syncthing-inotify: https://github.com/syncthing/syncthing-inotify/archive/v0.8.7.tar.gz
taylor: https://github.com/yopeso/Taylor/archive/0.2.2.tar.gz
tbb: https://github.com/01org/tbb/archive/2017_U7.tar.gz
terraform: https://github.com/hashicorp/terraform/archive/v0.10.4.tar.gz
terragrunt: https://github.com/gruntwork-io/terragrunt/archive/v0.13.2.tar.gz
treefrog: https://github.com/treefrogframework/treefrog-framework/archive/v1.18.0.tar.gz
voldemort: https://github.com/voldemort/voldemort/archive/release-1.10.25-cutoff.tar.gz
voltdb: https://github.com/VoltDB/voltdb/archive/voltdb-6.9.tar.gz
wolfssl: https://github.com/wolfSSL/wolfssl/archive/v3.12.0-stable.tar.gz
wrangler: https://github.com/RefactoringTools/wrangler/archive/wrangler1.2.tar.gz
xctool: https://github.com/facebook/xctool/archive/0.3.3.tar.gz
yaml-cpp: https://github.com/jbeder/yaml-cpp/archive/release-0.5.3.tar.gz
zorba: https://github.com/28msec/zorba/archive/3.1.tar.gz

Colorize as:
Show Line Numbers

Lisppaste pastes can be made by anyone at any time. Imagine a fearsomely comprehensive disclaimer of liability. Now fear, comprehensively.