Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2948 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.1510. Min Act: -0.1700

Data Index: 4377904 (Open Web Text)

Max Activating Token Index: 373

Text #1

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.0625. Min Act: -0.1700

Data Index: 6341191 (Open Web Text)

Max Activating Token Index: 419

Text #2

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.0125. Min Act: -0.1700

Data Index: 2396120 (Open Web Text)

Max Activating Token Index: 628

Text #3

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9916. Min Act: -0.1700

Data Index: 766412 (Open Web Text)

Max Activating Token Index: 384

Text #4

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9775. Min Act: -0.1700

Data Index: 5184201 (Open Web Text)

Max Activating Token Index: 552

Text #5

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9912. Min Act: -0.1700

Data Index: 3370168 (Open Web Text)

Max Activating Token Index: 90

Text #6

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9776. Min Act: -0.1700

Data Index: 7088902 (Open Web Text)

Max Activating Token Index: 861

Text #7

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9602. Min Act: -0.1700

Data Index: 6183522 (Open Web Text)

Max Activating Token Index: 294

Text #8

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9565. Min Act: -0.1700

Data Index: 37444 (Open Web Text)

Max Activating Token Index: 875

Text #9

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9334. Min Act: -0.1700

Data Index: 8611880 (Open Web Text)

Max Activating Token Index: 696

Text #10

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9318. Min Act: -0.1700

Data Index: 5966928 (Open Web Text)

Max Activating Token Index: 221

Text #11

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9270. Min Act: -0.1700

Data Index: 1565598 (Open Web Text)

Max Activating Token Index: 903

Text #12

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9272. Min Act: -0.1700

Data Index: 2043462 (Open Web Text)

Max Activating Token Index: 193

Text #13

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9124. Min Act: -0.1700

Data Index: 1461819 (Open Web Text)

Max Activating Token Index: 540

Text #14

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9185. Min Act: -0.1700

Data Index: 1482857 (Open Web Text)

Max Activating Token Index: 631

Text #15

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9043. Min Act: -0.1700

Data Index: 7307575 (Open Web Text)

Max Activating Token Index: 203

Text #16

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.8728. Min Act: -0.1700

Data Index: 3187939 (Open Web Text)

Max Activating Token Index: 100

Text #17

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.8990. Min Act: -0.1700

Data Index: 3913777 (Open Web Text)

Max Activating Token Index: 102

Text #18

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.8855. Min Act: -0.1700

Data Index: 6672572 (Open Web Text)

Max Activating Token Index: 497

Text #19

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.8882. Min Act: -0.1700

Data Index: 6759276 (Open Web Text)

Max Activating Token Index: 455