Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 774 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.6357. Min Act: -0.1700

Data Index: 8812347 (Open Web Text)

Max Activating Token Index: 700

Text #1

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.6395. Min Act: -0.1700

Data Index: 4396208 (Open Web Text)

Max Activating Token Index: 942

Text #2

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.6161. Min Act: -0.1700

Data Index: 2340795 (Open Web Text)

Max Activating Token Index: 106

Text #3

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.6137. Min Act: -0.1700

Data Index: 8238881 (Open Web Text)

Max Activating Token Index: 192

Text #4

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5593. Min Act: -0.1700

Data Index: 745246 (Open Web Text)

Max Activating Token Index: 44

Text #5

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5363. Min Act: -0.1700

Data Index: 8035810 (Open Web Text)

Max Activating Token Index: 182

Text #6

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5271. Min Act: -0.1700

Data Index: 1298888 (Open Web Text)

Max Activating Token Index: 97

Text #7

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4849. Min Act: -0.1700

Data Index: 745245 (Open Web Text)

Max Activating Token Index: 370

Text #8

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5010. Min Act: -0.1700

Data Index: 6577253 (Open Web Text)

Max Activating Token Index: 241

Text #9

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5000. Min Act: -0.1700

Data Index: 699078 (Open Web Text)

Max Activating Token Index: 365

Text #10

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.5059. Min Act: -0.1700

Data Index: 7364023 (Open Web Text)

Max Activating Token Index: 276

Text #11

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4662. Min Act: -0.1700

Data Index: 4305039 (Open Web Text)

Max Activating Token Index: 882

Text #12

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4706. Min Act: -0.1700

Data Index: 728254 (Open Web Text)

Max Activating Token Index: 246

Text #13

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4535. Min Act: -0.1700

Data Index: 8344158 (Open Web Text)

Max Activating Token Index: 977

Text #14

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4604. Min Act: -0.1700

Data Index: 7143075 (Open Web Text)

Max Activating Token Index: 888

Text #15

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4451. Min Act: -0.1700

Data Index: 1295419 (Open Web Text)

Max Activating Token Index: 689

Text #16

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4264. Min Act: -0.1700

Data Index: 1605809 (Open Web Text)

Max Activating Token Index: 63

Text #17

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4347. Min Act: -0.1700

Data Index: 3258097 (Open Web Text)

Max Activating Token Index: 275

Text #18

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4800. Min Act: -0.1700

Data Index: 3391944 (Open Web Text)

Max Activating Token Index: 287

Text #19

Max Range: 4.6395. Min Range: -4.6395

Max Act: 4.4699. Min Act: -0.1700

Data Index: 3856132 (Open Web Text)

Max Activating Token Index: 293