https://rakyll.org/ Recent content on rakyll.org Hugo -- gohugo.io en-us rakyll@rakyll.org (Jaana Dogan) rakyll@rakyll.org (Jaana Dogan) Wed, 27 Dec 2017 11:27:27 -0400 https://rakyll.org/generics-facilititators/ Mon, 20 Dec 2021 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/generics-facilititators/ Go 1.18 is going to be released with generics support. Adding generics to Go was a multi-year effort and was a difficult one. Go type system is not a traditional type system and it was not possible just to bring an existing generics implementation from other language and be done. The current proposal was accepted after years of user research, experiments and discussions. The proposal got iterated a few times during the implementation phase. https://rakyll.org/shardz/ Fri, 03 Dec 2021 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/shardz/ Shard coordination has been one of the bigger challenges to design sharded systems especially for engineers with little experience in the subject. Companies like Facebook have been using general purpose shard coordinators, e.g. Shard Manager, and suggesting that general purpose sharding policies have been widely successful. A general purpose sharding coordinator is not a solution to all advanced sharding needs, but a starting point for average use cases. It’s a framework to think about core abstractions in sharded systems and providing a protocol that orchestrate sharding decisions. https://rakyll.org/spanner-sql/ Wed, 22 Jul 2020 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/spanner-sql/ Spanner is a distributed database Google initiated a while ago to build a highly available and highly consistent database for its own workloads. Spanner was initially built to be a key/value and was in a completely different shape than it is today and it had different goals. Since the beginning, it had transactional capabilities, external consistency and was able to failover transparently. Over time, Spanner adopted a strongly typed schema and some other relational database features. https://rakyll.org/generics-proposal/ Wed, 17 Jun 2020 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/generics-proposal/ Update: The proposal draft has been revisited to use brackets instead of parenthesis. This article will be updated with the new syntax soon. Ian Lance Taylor and Robert Griesemer have been working on a generics proposal for Go for a while. Unlike other proposals, a highly significant language change as such generics will require experimentation and comprehensive feedback before it can be finalized and submitted as a formal language change proposal. https://rakyll.org/spanner-ha-writes/ Tue, 19 May 2020 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/spanner-ha-writes/ Spanner is a relational database with 99.999% availability which is roughly 5 mins a year. Spanner is a distributed system and can span multiple machines, multiple datacenters (and even geographical regions when configured). It splits the records automatically among its replicas and provides automatic failover. Unlike traditional failover models, Spanner doesn’t failover to a secondary cluster but can elect an available read-write replica as the new leader. In relational databases, providing both high availability and high consistency in writes is a very hard problem. https://rakyll.org/go-cloud/ Thu, 06 Feb 2020 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/go-cloud/ Go has been continuously growing in the past decade, especially among the infrastructure teams and in the cloud ecosystem. In this article, we will go through some of the unique strengths of Go in this field. We will also cover some gotchas that may not be obvious to the users at the first sight. Build small binaries. Go builds small binaries. This makes it a good language to build artifacts for containerized or serverless environments. https://rakyll.org/inlined-defers/ Mon, 20 Jan 2020 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/inlined-defers/ Go’s defer keyword allows us to schedule a function to run before a function returns. Multiple functions can be deferred from a function. defer is often used to cleanup resources, finish function-scoped tasks, and similar. Deferring functions are great for maintability. By deferring, for example, we reduce the risk of forgetting to close the file in the rest of the program: func main() { f, err := os.Open("hello.txt") if err ! https://rakyll.org/numa/ Mon, 18 Nov 2019 11:28:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/numa/ Non-uniform memory access (NUMA) is an approach to optimize memory access time in multi-processor architectures. In NUMA architectures, processors can access to the memory chips near them instead of going to the physically distant ones. In the distant past CPUs generally ran slower than the memory. Today, CPUs are quite faster than the memory they use. This became a problem because processors constantly started to wait for data to be retrieved from the memory. https://rakyll.org/typesystem/ Wed, 27 Dec 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/typesystem/ It is real struggle to work with a new language, especially if the type doesn’t resemble what you have previously seen. I have been there with Go and lost my interest in the language when it first came out due to the reason I was pretending it is something I already knew. Go is considered as an object-oriented language even though it lacks type hierarchy. It has an unconventional type system. https://rakyll.org/pprof-ui/ Tue, 10 Oct 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/pprof-ui/ pprof now is coming with a Web UI. In order to try it out, go get the pprof tool: $ go get github.com/google/pprof The tool launches a web UI if -http flag is provided. For example, in order to launch the UI with an existing profile data, run the following command: $ pprof -http=:8080 profile.out You can focus, ignore, hide, and show with regexp. As well as clicking on the boxes and using the refining menu also works: https://rakyll.org/latency-profiling/ Tue, 18 Jul 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/latency-profiling/ Note: This article contains non-finalized ideas; we may end up not implementing any of this but ideally we should do work towards the direction explained here. Go is the language to write servers, Go is the language to write microservices. Yet, we haven’t done much in the past for latency analysis and observability/diagnostics of request/RPC performance. GopherCon 2017 was an opportunity for me to discuss our roadmap for latency analysis. I have talked to many whose main job is to provide instrumentation solutions to the ecosystem. https://rakyll.org/scheduler/ Sun, 16 Jul 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/scheduler/ Go scheduler’s job is to distribute runnable goroutines over multiple worker OS threads that runs on one or more processors. In multi-threaded computation, two paradigms have emerged in scheduling: work sharing and work stealing. Work-sharing: When a processor generates new threads, it attempts to migrate some of them to the other processors with the hopes of them being utilized by the idle/underutilized processors. Work-stealing: An underutilized processor actively looks for other processor’s threads and “steal” some. https://rakyll.org/profiler-labels/ Mon, 03 Jul 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/profiler-labels/ Go 1.9 is introducing profiler labels, a way to add arbitrary key-values to the samples collected by the CPU profiler. CPU profilers collect and output hot spots where the CPU spent most time in when executing. A typical CPU profiler output is primarily reports the location of these spots as function name, source file/line, etc. By looking at the data, you can also examine which parts of the code invoked these spots. https://rakyll.org/custom-profiles/ Fri, 30 Jun 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/custom-profiles/ Go provides several pprof profiles out of thet box to gather profiling data from Go programs. The builtin profiles provided by the runtime/pprof package: profile: CPU profile determines where a program spends its time while actively consuming CPU cycles (as opposed while sleeping or waiting for I/O). heap: Heap profile reports the currently live allocations; used to monitor current memory usage or check for memory leaks. threadcreate: Thread creation profile reports the sections of the program that lead the creation of new OS threads. https://rakyll.org/coredumps/ Mon, 22 May 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/coredumps/ Debugging is highly useful to examine the execution flow and to understand the current state of a program. A core file is a file that contains the memory dump of a running process and its process status. It is primarily used for post-mortem debugging of a program, as well as to understand a program’s state while it is still running. These two cases make debugging of core dumps a good diagnostics aid to postmortem and analyze production services. https://rakyll.org/grpc-trace/ Wed, 22 Mar 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/grpc-trace/ In monolithic systems, it is relatively easy to collect diagnostic data from the building blocks of a program. All modules live within one process and share common resources to report logs and errors. Once you are distributing your system into microservices, it becomes harder to follow a call starting from the user’s entry point until a response is served. To address this problem, Google invented Dapper to instrument and analyze its production services. https://rakyll.org/naming-tests-to-doc/ Fri, 03 Feb 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/naming-tests-to-doc/ Go doesn’t specifically enforce you how you choose your test names. Tests are a significant contributors for the maintainability of your code. Tests not just providing correctness checking but also are useful in self documenting your code and its usage. On top of that, tests are the single best source to read about responsibilities of a type, function, etc. This is where naming tests better can help the internal documentation and therefore the maintainability of your code. https://rakyll.org/style-packages/ Sat, 14 Jan 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/style-packages/ Go is about naming and organization as much as everything else in the language. Well-organized Go code is easy to discover, use and read. Well-organized code is as critical as well designed APIs. The location, name, and the structure of your packages are the first elements your users see and interact with. This document’s goal is to guide you with common good practices not to set rules. You will always need to use your own judgement to pick the most elegant solution for your specific case. https://rakyll.org/go18cls/ Wed, 11 Jan 2017 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/go18cls/ Go 1.8 is going to to launched in February 2017. There is a sizable list of new features and improvements on the release notes. While these notes is the best summary to see what has happened in the last 6 months, I will try to give you some stats to give you a sense of the size of the work. I have examined all the changes merged into the tree during the Go 1. https://rakyll.org/mutexprofile/ Mon, 19 Dec 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/mutexprofile/ Go 1.8 introduces a new profile, the contended mutex profile, that allows you to capture a fraction of the stack traces of goroutines with contended mutexes. You need to set the sampling fraction by calling runtime.SetMutexProfileFraction to a value above zero to enable collection. Consider the following program: import _ "net/http/pprof" var mu sync.Mutex var items = make(map[int]struct{}) runtime.SetMutexProfileFraction(5) for i := 0; i < 1000*1000; i++ { go func(i int) { mu. https://rakyll.org/default-gopath/ Wed, 14 Dec 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/default-gopath/ Go 1.8 will set a default GOPATH if the GOPATH env variable is not set. The requirement of setting a GOPATH has been a major issue for Go users who installed the Go tools for the first time and got the “you have to set a GOPATH” error in their initial experience with the tools. Explaining the GOPATH is and instructing how to set this env variable were both distracting new users away from using Go. https://rakyll.org/http2push/ Sat, 10 Dec 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/http2push/ Go 1.8 is going to feature support for HTTP/2 server push. HTTP/2 has many features designed to make the Web faster. One of those features is the server push, the ability to send resources before the client asks for it. This feature enables websites to push assets like JavaScript and CSS files before waiting for the web page to be loaded and asking for those resources. net/http package will support server push by exposing a Pusher API that will be supported by HTTP/2 ResponseWriters. https://rakyll.org/deprecated/ Thu, 08 Dec 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/deprecated/ In Go, for a long time, we didn’t have a convention to label the deprecated APIs. In the past years, there is new convention emerged to add deprecation notices to the docs. Today, standard library uses this specific format. As an example, Go 1.8 deprecates sql/driver.Execer and adds a deprecation notice to its godoc. // Execer is an optional interface that may be implemented by a Conn. // // If a Conn does not implement Execer, the sql package's DB. https://rakyll.org/leakingctx/ Tue, 25 Oct 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/leakingctx/ The context package makes it possible to manage a chain of calls within the same call path by signaling context’s Done channel. In this article, we will examine how to use the context package to avoid leaking goroutines. Assume, you have a function that starts a goroutine internally. Once this function is called, the caller may not be able to terminate the goroutine started by the function. // gen is a broken generator that will leak a goroutine. https://rakyll.org/codegen/ Sat, 15 Oct 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/codegen/ Last week, I was at dotGo. I gave a very short lightning talk about inspection of code generation with the tools already available in the toolchain. This post goes through the talk for those who didn’t have the privilege to be at the conference. Slides are also available at go-talks. Throughout this article, we will use the following program: package main import "fmt" func main() { sum := 1 + 1 fmt. https://rakyll.org/human-first/ Tue, 11 Oct 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/human-first/ Disclaimer: I forked my opinions on this one from a barely readable Twitter thread and wanted to write it down how I feel about keeping the language internals away from the users, especially from the newcomers. This is not a skill-level concern, it is a core goal of Go to provide a high-level programming language that saves users from excessive mental overhead. Note that these are personal opinions and are not written on the behalf of a group. https://rakyll.org/go-tool-flags/ Sun, 25 Sep 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/go-tool-flags/ New to the Go tools? Or do you want to expand your knowledge? This article is about the flags for the Go tools everyone should know. Disclaimer: This article might be slightly biased. This is a collection of flags I personally use and flags people around me having trouble finding references for. If you have more ideas, ping me on Twitter. $ go build -x -x lists all the commands go build invokes. https://rakyll.org/examples-coverage/ Thu, 08 Sep 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/examples-coverage/ Go programming language provides many unique good features to write and maintain examples for your packages backed by the testing tools. As an addition to the test coverage and test coverage report, go test also can provide coverage for testable examples. Use the following commands in your package to use the -run flag to only the match the example tests and view the results in your browser. $ go test -v -run=Example -coverprofile=c. https://rakyll.org/parallelize-test-tables/ Tue, 06 Sep 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/parallelize-test-tables/ With Go 1.7, testing package supports sub-tests that allows you to run multiple smaller tests from a test case. Each sub test is reported independently in the go test output. More information about these recent additions can be found at Marcel van Lohuizen’s recent talk from GolangUK 2016. These additions to Go 1.7 enabled reporting and other testing.T functionality for subtests. One of the biggest contributions of the recent changes is to be able to use these features for table-driven tests. https://rakyll.org/instruments/ Fri, 02 Sep 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/instruments/ Apple has a suite of instrumentation and tracing tools for performance analysis available as a part of their Xcode tooling set. In this article, we will use Instruments to record and analyze the CPU profile of a Go program. Instruments also provide a large set of macOS-specific tracing and profiling if you have performance issues specifically on darwin. Some of these specific profiles are: System trace: Collects comprehensive information about system calls, scheduling, user-kernel space transitions. https://rakyll.org/grpc-streaming/ Tue, 30 Aug 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/grpc-streaming/ Disclaimer: This article is not about a core Go package or tool but gRPC. gRPC provides support for implementing streaming endpoints as well as streaming support in their clients. Bidirectional streaming is useful if you want both server and client to be able to communicate to the other side independently in a full duplex fashion. In this article, I will dig into how to use the streaming gRPC Go client to talk to a streaming API endpoint. https://rakyll.org/about/ Sat, 27 Aug 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/about/ About the author I work at Google working on our AI infrastructure. I’ve spent many years at Google where I worked on Google Drive, the API infrastructure, Go Programming Language, and our monitoring stack. I worked for smaller companies building for years lead designing and operating various systems. I live in San Francisco, CA and I’m a polyglot. Contact Email: jbd@google.com, rakyll@rakyll.org Github: rakyll Twitter: rakyll https://rakyll.org/eg/ Sat, 27 Aug 2016 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/eg/ If you are willing to make large scale refactoring in your Go programs, automating the refactoring tasks is more desirable than manual editing. eg is a program that allows you to perform transformations based on template Go files. To install the tool, run the following: $ go get golang.org/x/tools/cmd/eg eg requires a template file to look for which transformation it should apply to your source code. What’s nice is that the template file is a Go file with little annotations. https://rakyll.org/swift/ Sat, 03 Oct 2015 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/swift/ Note: Swift bindings are highly experimental and subject to change. This work must currently be classified as preliminary work and we will be improving APIs in the long term. As a part of the Go Mobile, we have announced tools and packages that make language bindings from Java to Go and Objective-C to Go available. A relatively new and less documented aspect of the bindings is the availability of the Swift to Go calls. https://rakyll.org/cross-compilation/ Tue, 08 Sep 2015 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/cross-compilation/ Note: This article extends Dave Cheney’s Go 1.5 cross compilers post. Cross compilers got easier with Go 1.5. You don’t have to bootstrap the standard library and toolchain as you used to do prior to 1.5. If cgo is not required The go tool won’t require any bootstrapping if cgo is not required. That allows you to target the following program to any GOOS/GOARCH without requiring you to do any additional work. https://rakyll.org/interface-pollution/ Sat, 18 Oct 2014 11:27:27 -0400 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/interface-pollution/ If there was a single powerful language feature in Go, it’d be the interfaces. The internals of Go contain strong combinations of useful ideas from various type systems and inevitably they ring the curiosity bells. I recently surveyed Github for Go interface declarations, and the results indicated that Go users pollute the environment with interfaces no one needs or will use. Don’t export any interfaces until you have to. Interfaces are great, but interface pollution is not so. https://rakyll.org/archive/ Mon, 01 Jan 0001 00:00:00 +0000 rakyll@rakyll.org (Jaana Dogan) https://rakyll.org/archive/