Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1750 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.2241. Min Act: -0.1700

Data Index: 2262030 (Open Web Text)

Max Activating Token Index: 830

Text #1

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.1886. Min Act: -0.1700

Data Index: 1218161 (Open Web Text)

Max Activating Token Index: 532

Text #2

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.2173. Min Act: -0.1700

Data Index: 3624679 (Open Web Text)

Max Activating Token Index: 388

Text #3

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.1307. Min Act: -0.1700

Data Index: 8308958 (Open Web Text)

Max Activating Token Index: 869

Text #4

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.1354. Min Act: -0.1700

Data Index: 1674057 (Open Web Text)

Max Activating Token Index: 681

Text #5

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0857. Min Act: -0.1700

Data Index: 6039746 (Open Web Text)

Max Activating Token Index: 81

Text #6

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0808. Min Act: -0.1700

Data Index: 7292732 (Open Web Text)

Max Activating Token Index: 724

Text #7

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0501. Min Act: -0.1700

Data Index: 4172225 (Open Web Text)

Max Activating Token Index: 418

Text #8

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0706. Min Act: -0.1700

Data Index: 6253504 (Open Web Text)

Max Activating Token Index: 526

Text #9

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0671. Min Act: -0.1700

Data Index: 6796027 (Open Web Text)

Max Activating Token Index: 991

Text #10

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0208. Min Act: -0.1700

Data Index: 7800135 (Open Web Text)

Max Activating Token Index: 699

Text #11

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0323. Min Act: -0.1700

Data Index: 7315659 (Open Web Text)

Max Activating Token Index: 242

Text #12

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0197. Min Act: -0.1700

Data Index: 3564330 (Open Web Text)

Max Activating Token Index: 837

Text #13

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0152. Min Act: -0.1700

Data Index: 6781627 (Open Web Text)

Max Activating Token Index: 188

Text #14

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0056. Min Act: -0.1700

Data Index: 362423 (Open Web Text)

Max Activating Token Index: 392

Text #15

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0005. Min Act: -0.1700

Data Index: 755327 (Open Web Text)

Max Activating Token Index: 990

Text #16

Max Range: 3.2241. Min Range: -3.2241

Max Act: 2.9919. Min Act: -0.1700

Data Index: 3218865 (Open Web Text)

Max Activating Token Index: 561

Text #17

Max Range: 3.2241. Min Range: -3.2241

Max Act: 2.9935. Min Act: -0.1700

Data Index: 7315385 (Open Web Text)

Max Activating Token Index: 893

Text #18

Max Range: 3.2241. Min Range: -3.2241

Max Act: 3.0067. Min Act: -0.1700

Data Index: 7573166 (Open Web Text)

Max Activating Token Index: 788

Text #19

Max Range: 3.2241. Min Range: -3.2241

Max Act: 2.9861. Min Act: -0.1700

Data Index: 329439 (Open Web Text)

Max Activating Token Index: 545