golang依賴注入db,golang依賴注入什麼意思

本文目錄一覽：

• 1、go依賴注入dig包使用-來自uber公司
• 2、golang配製高性能sql.DB
• 3、golang私有倉庫依賴配置

go依賴注入dig包使用-來自uber公司

原文鏈接：

github:

Dependency Injection is the idea that your components (usually structs in go) should receive their dependencies when being created. This runs counter to the associated anti-pattern of components building their own dependencies during initialization. Let’s look at an example.

Suppose you have a Server struct that requires a Config struct to implement its behavior. One way to do this would be for the Server to build its own Config during initialization.

This seems convenient. Our caller doesn’t have to be aware that our Server even needs access to Config . This is all hidden from the user of our function.

However, there are some disadvantages. First of all, if we want to change the way our Config is built, we’ll have to change all the places that call the building code. Suppose, for example, our buildMyConfigSomehow function now needs an argument. Every call site would need access to that argument and would need to pass it into the building function.

Also, it gets really tricky to mock the behavior of our Config . We’ll somehow have to reach inside of our New function to monkey with the creation of Config .

Here’s the DI way to do it:

Now the creation of our Server is decoupled from the creation of the Config . We can use whatever logic we want to create the Config and then pass the resulting data to our New function.

Furthermore, if Config is an interface, this gives us an easy route to mocking. We can pass anything we want into New as long as it implements our interface. This makes testing our Server with mock implementations of Config simple.

The main downside is that it’s a pain to have to manually create the Config before we can create the Server . We’ve created a dependency graph here – we must create our Config first because of Server depends on it. In real applications these dependency graphs can become very large and this leads to complicated logic for building all of the components your application needs to do its job.

This is where DI frameworks can help. A DI framework generally provides two pieces of functionality:

A DI framework generally builds a graph based on the “providers” you tell it about and determines how to build your objects. This is very hard to understand in the abstract, so let’s walk through a moderately-sized example.

We’re going to be reviewing the code for an HTTP server that delivers a JSON response when a client makes a GET request to /people . We’ll review the code piece by piece. For simplicity sake, it all lives in the same package ( main ). Please don’t do this in real Go applications. Full code for this example can be found here .

First, let’s look at our Person struct. It has no behavior save for some JSON tags.

A Person has an Id , Name and Age . That’s it.

Next let’s look at our Config . Similar to Person , it has no dependencies. Unlike Person , we will provide a constructor.

Enabled tells us if our application should return real data. DatabasePath tells us where our database lives (we’re using sqlite). Port tells us the port on which we’ll be running our server.

Here’s the function we’ll use to open our database connection. It relies on our Config and returns a *sql.DB .

Next we’ll look at our PersonRepository . This struct will be responsible for fetching people from our database and deserializing those database results into proper Person structs.

PersonRepository requires a database connection to be built. It exposes a single function called FindAll that uses our database connection to return a list of Person structs representing the data in our database.

To provide a layer between our HTTP server and the PersonRepository , we’ll create a PersonService .

Our PersonService relies on both the Config and the PersonRepository . It exposes a function called FindAll that conditionally calls the PersonRepository if the application is enabled.

Finally, we’ve got our Server . This is responsible for running an HTTP server and delegating the appropriate requests to our PersonService .

The Server is dependent on the PersonService and the Config .

Ok, we know all the components of our system. Now how the hell do we actually initialize them and start our system?

First, let’s write our main() function the old fashioned way.

First, we create our Config . Then, using the Config , we create our database connection. From there we can create our PersonRepository which allows us to create our PersonService . Finally, we can use this to create our Server and run it.

Phew, that was complicated. Worse, as our application becomes more complicated, our main will continue to grow in complexity. Every time we add a new dependency to any of our components, we’ll have to reflect that dependency with ordering and logic in the main function to build that component.

As you might have guessed, a Dependency Injection framework can help us solve this problem. Let’s examine how.

The term “container” is often used in DI frameworks to describe the thing into which you add “providers” and out of which you ask for fully-build objects. The dig library gives us the Provide function for adding providers and the Invoke function for retrieving fully-built objects out of the container.

First, we build a new container.

Now we can add new providers. To do so, we call the Provide function on the container. It takes a single argument: a function. This function can have any number of arguments (representing the dependencies of the component to be created) and one or two return values (representing the component that the function provides and optionally an error).

The above code says “I provide a Config type to the container. In order to build it, I don’t need anything else.” Now that we’ve shown the container how to build a Config type, we can use this to build other types.

This code says “I provide a *sql.DB type to the container. In order to build it, I need a Config . I may also optionally return an error.”

In both of these cases, we’re being more verbose than necessary. Because we already have NewConfig and ConnectDatabase functions defined, we can use them directly as providers for the container.

Now, we can ask the container to give us a fully-built component for any of the types we’ve provided. We do so using the Invoke function. The Invoke function takes a single argument – a function with any number of arguments. The arguments to the function are the types we’d like the container to build for us.

The container does some really smart stuff. Here’s what happens:

That’s a lot of work the container is doing for us. In fact, it’s doing even more. The container is smart enough to build one, and only one, instance of each type provided. That means we’ll never accidentally create a second database connection if we’re using it in multiple places (say multiple repositories).

Now that we know how the dig container works, let’s use it to build a better main.

The only thing we haven’t seen before here is the error return value from Invoke . If any provider used by Invoke returns an error, our call to Invoke will halt and that error will be returned.

Even though this example is small, it should be easy to see some of the benefits of this approach over our “standard” main. These benefits become even more obvious as our application grows larger.

One of the most important benefits is the decoupling of the creation of our components from the creation of their dependencies. Say, for example, that our PersonRepository now needs access to the Config . All we have to do is change our NewPersonRepository constructor to include the Config as an argument. Nothing else in our code changes.

Other large benefits are lack of global state, lack of calls to init (dependencies are created lazily when needed and only created once, obviating the need for error-prone init setup) and ease of testing for individual components. Imagine creating your container in your tests and asking for a fully-build object to test. Or, create an object with mock implementations of all dependencies. All of these are much easier with the DI approach.

I believe Dependency Injection helps build more robust and testable applications. This is especially true as these applications grow in size. Go is well suited to building large applications and has a great DI tool in dig . I believe the Go community should embrace DI and use it in far more applications.

golang配製高性能sql.DB

有很多教程是關於Go的sql.DB類型和如何使用它來執行SQL數據庫查詢的。但大多數內容都沒有講述 SetMaxOpenConns() , SetMaxIdleConns() 和 SetConnMaxLifetime()方法， 您可以使用它們來配置sql.DB的行為並改變其性能。

轉自：

整理：go語言中文文檔：

在本文我將詳細解釋這些設置的作用，並說明它們所能產生的(積極和消極)影響。

一個sql.DB對象就是一個數據庫連接池，它包含“正在用”和“空閑的”連接。一個正在用的連接指的是，你正用它來執行數據庫任務，例如執行SQL語句或行查詢。當任務完成連接就是空閑的。

當您創建sql.DB執行數據庫任務時，它將首先檢查連接池中是否有可用的空閑連接。如果有可用的連接，那麼Go將重用現有連接，並在執行任務期間將其標記為正在使用。如果池中沒有空閑連接，而您需要一個空閑連接，那麼Go將創建一個新的連接。

默認情況下，在同一時間打開連接的數量是沒有限制(包含使用中+空閑)。但你可以通過SetMaxOpenConns()方法實現自定義限制，如下所示:

在這個示例代碼中，連接池現在有5個並發打開的連接數。如果所有5個連接都已經被標記為正在使用，並且需要另一個新的連接，那麼應用程序將被迫等待，直到5個連接中的一個被釋放並變為空閑。

為了說明更改MaxOpenConns的影響，我運行了一個基準測試，將最大打開連接數設置為1、2、5、10和無限。基準測試在PostgreSQL數據庫上執行並行的INSERT語句，您可以在這裡找到代碼。測試結果:

對於這個基準測試，我們可以看到，允許打開的連接越多，在數據庫上執行INSERT操作所花費的時間就越少(打開的連接數為1時，執行速度3129633ns/op，而無限連接：531030ns/op——大約快了6倍)。這是因為允許打開的連接越多，可以並發執行的數據庫查詢就越多。

默認情況下，sql.DB允許連接池中最多保留2個空閑連接。你可以通過SetMaxIdleConns()方法改變它，如下所示:

從理論上講，允許池中有更多的空閑連接將提高性能，因為這樣就不太可能從頭開始建立新連接——因此有助於提升數據庫性能。

讓我們來看看相同的基準測試，最大空閑連接設置為none, 1,2,5和10:

當MaxIdleConns設置為none時，必須為每個INSERT從頭創建一個新的連接，我們可以從基準測試中看到，平均運行時和內存使用量相對較高。

只允許保留和重用一個空閑連接對基準測試影響特別明顯——它將平均運行時間減少了大約8倍，內存使用量減少了大約20倍。繼續增加空閑連接池的大小會使性能變得更好，儘管改進並不明顯。

那麼，您應該維護一個大的空閑連接池嗎?答案取決於應用程序。重要的是要意識到保持空閑連接是有代價的—它佔用了可以用於應用程序和數據庫的內存。

還有一種可能是，如果一個連接空閑時間太長，那麼它可能會變得不可用。例如，MySQL的wait_timeout設置將自動關閉任何8小時(默認)內未使用的連接。

當發生這種情況時，sql.DB會優雅地處理它。壞連接將自動重試兩次，然後放棄，此時Go將該連接從連接池中刪除，並創建一個新的連接。因此，將MaxIdleConns設置得太大可能會導致連接變得不可用，與空閑連接池更小(使用更頻繁的連接更少)相比，會佔有更多的資源。所以，如果你很可能很快就會再次使用，你只需保持一個空閑的連接。

最後要指出的是，MaxIdleConns應該總是小於或等於MaxOpenConns。Go強制執行此操作，並在必要時自動減少MaxIdleConns。

現在讓我們看看SetConnMaxLifetime()方法，它設置連接可重用的最大時間長度。如果您的SQL數據庫也實現了最大連接生命周期，或者—例如—您希望方便地在負載均衡器後交換數據庫，那麼這將非常有用。

你可以這樣使用它:

在這個例子中，所有的連接都將在創建後1小時“過期”，並且在過期後無法重用。但注意:

從理論上講，ConnMaxLifetime越短，連接過期的頻率就越高——因此，需要從頭創建連接的頻率就越高。為了說明這一點，我運行了將ConnMaxLifetime設置為100ms、200ms、500ms、1000ms和無限(永遠重用)的基準測試，默認設置為無限打開連接和2個空閑連接。這些時間段顯然比您在大多數應用程序中使用的時間要短得多，但它們有助於很好地說明行為。

在這些特定的基準測試中，我們可以看到，與無限生存期相比，在100ms生存期時內存使用量增加了3倍以上，而且每個INSERT的平均運行時也稍微長一些。

如果您在代碼中設置了ConnMaxLifetime，那麼一定要記住連接將過期(隨後重新創建)的頻率。例如，如果您總共有100個連接，而ConnMaxLifetime為1分鐘，那麼您的應用程序可能每秒鐘殺死和重新創建1.67個連接(平均值)。您不希望這個頻率太大，最終會阻礙性能，而不是提高性能。

最後，如果不說明超過數據庫連接數量的硬限制將會發生什麼，那麼本文就不完整了。 為了說明這一點，我將修改postgresql.conf文件，這樣總共只允許5個連接(默認是100個)…

然後在無限連接的情況下重新運行基準測試……

一旦達到5個連接的硬限制，數據庫驅動程序(pq)立即返回一個太多客戶端連接的錯誤消息，而無法完成INSERT。為了防止這個錯誤，我們需要將sql.DB中打開連接的最大總數(正在使用的+空閑的)設置為低於5。像這樣:

現在，sql.DB在任何時候最多只能創建3個連接，基準測試運行時應該不會出現任何錯誤。但是這樣做需要注意：當達到開放連接數限制，並且所有連接都在使用時，應用程序需要執行的任何新的數據庫任務都將被迫等待，直到連接標記為空閑。例如，在web應用程序的上下文中，用戶的HTTP請求看起來會“掛起”，甚至在等待數據庫任務運行時可能會超時。

為了減輕這種情況，你應該始終在一個上下文中傳遞。在調用數據庫時，啟用上下文的方法(如ExecContext())，使用固定的、快速的超時上下文對象。

總結

1、根據經驗，應該顯式設置MaxOpenConns值。這應該小於數據庫和基礎設施對連接數量的硬性限制。

2、一般來說，更高的MaxOpenConns和MaxIdleConns值將帶來更好的性能。但你應該注意到效果是遞減的，連接池空閑連接太多(連接沒有被重用，最終會變壞)實際上會導致性能下降。

3、為了降低上面第2點帶來的風險，您可能需要設置一個相對較短的ConnMaxLifetime。但你也不希望它太短，導致連接被殺死或不必要地頻繁重建。

4、MaxIdleConns應該總是小於或等於MaxOpenConns。

對於中小型web應用程序，我通常使用以下設置作為起點，然後根據實際吞吐量水平的負載測試結果進行優化。

golang私有倉庫依賴配置

golang私有倉庫依賴配置

    版本要求：go 1.14+

    go mod 配置：

        go env -w GOPRIVATE=”gitlab.xxx.com”    //配置私有倉庫域名 ：重要

        go env -w GONOPROXY=”gitlab.xxx.com”    //此配置下的域名默認不走代理

        go env -w GONOSUMDB=”gitlab.xxx.com”    //此配置下的域名默認不進行gosumdb校驗

        go env -w GOINSECURE=”gitlab.xxx.com”    //此配置下的域名默認採用http協議 。有些公司的私有倉庫採用http協議，而go mod默認採用                                                                                       https，請根據實際情況進行配置：重要

    賬戶及密碼：

        因私有倉庫一般都需要進行登錄，所以可以通過隱藏文件進行用戶名及密碼配置。

        文件路徑：~/.netrc    //默認Linux系統，

        文件內容：

            machine 域名    //gitlab.xxx.com

            login 賬號

            password 密碼