Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1919 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.1365. Min Act: -0.1700

Data Index: 7951213 (Open Web Text)

Max Activating Token Index: 1020

Text #1

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.1082. Min Act: -0.1700

Data Index: 1891785 (Open Web Text)

Max Activating Token Index: 227

Text #2

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.0711. Min Act: -0.1700

Data Index: 8267625 (Open Web Text)

Max Activating Token Index: 118

Text #3

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.0189. Min Act: -0.1700

Data Index: 3331653 (Open Web Text)

Max Activating Token Index: 735

Text #4

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.0317. Min Act: -0.1700

Data Index: 4595773 (Open Web Text)

Max Activating Token Index: 62

Text #5

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.0030. Min Act: -0.1700

Data Index: 2279765 (Open Web Text)

Max Activating Token Index: 55

Text #6

Max Range: 3.1365. Min Range: -3.1365

Max Act: 3.0010. Min Act: -0.1700

Data Index: 2646649 (Open Web Text)

Max Activating Token Index: 739

Text #7

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9932. Min Act: -0.1700

Data Index: 7730136 (Open Web Text)

Max Activating Token Index: 786

Text #8

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9637. Min Act: -0.1700

Data Index: 2572799 (Open Web Text)

Max Activating Token Index: 989

Text #9

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9468. Min Act: -0.1700

Data Index: 6614488 (Open Web Text)

Max Activating Token Index: 1019

Text #10

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9083. Min Act: -0.1700

Data Index: 2093191 (Open Web Text)

Max Activating Token Index: 273

Text #11

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8943. Min Act: -0.1700

Data Index: 423739 (Open Web Text)

Max Activating Token Index: 560

Text #12

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8981. Min Act: -0.1700

Data Index: 8159885 (Open Web Text)

Max Activating Token Index: 575

Text #13

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9098. Min Act: -0.1700

Data Index: 5348181 (Open Web Text)

Max Activating Token Index: 380

Text #14

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.9116. Min Act: -0.1700

Data Index: 2104623 (Open Web Text)

Max Activating Token Index: 319

Text #15

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8884. Min Act: -0.1700

Data Index: 8060201 (Open Web Text)

Max Activating Token Index: 757

Text #16

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8903. Min Act: -0.1700

Data Index: 7867515 (Open Web Text)

Max Activating Token Index: 519

Text #17

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8965. Min Act: -0.1700

Data Index: 852901 (Open Web Text)

Max Activating Token Index: 437

Text #18

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8896. Min Act: -0.1700

Data Index: 8061231 (Open Web Text)

Max Activating Token Index: 676

Text #19

Max Range: 3.1365. Min Range: -3.1365

Max Act: 2.8851. Min Act: -0.1700

Data Index: 3866933 (Open Web Text)

Max Activating Token Index: 221