Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 295 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.5046. Min Act: -0.1700

Data Index: 241925 (Open Web Text)

Max Activating Token Index: 845

Text #1

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.2542. Min Act: -0.1700

Data Index: 1221506 (Open Web Text)

Max Activating Token Index: 433

Text #2

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.2884. Min Act: -0.1700

Data Index: 7737191 (Open Web Text)

Max Activating Token Index: 1014

Text #3

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.2283. Min Act: -0.1700

Data Index: 5469123 (Open Web Text)

Max Activating Token Index: 742

Text #4

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.2095. Min Act: -0.1700

Data Index: 3948636 (Open Web Text)

Max Activating Token Index: 568

Text #5

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1912. Min Act: -0.1700

Data Index: 357370 (Open Web Text)

Max Activating Token Index: 558

Text #6

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.2238. Min Act: -0.1700

Data Index: 4663626 (Open Web Text)

Max Activating Token Index: 447

Text #7

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1921. Min Act: -0.1700

Data Index: 1221504 (Open Web Text)

Max Activating Token Index: 869

Text #8

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1949. Min Act: -0.1700

Data Index: 4057482 (Open Web Text)

Max Activating Token Index: 442

Text #9

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1807. Min Act: -0.1700

Data Index: 4382372 (Open Web Text)

Max Activating Token Index: 188

Text #10

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1559. Min Act: -0.1700

Data Index: 4406994 (Open Web Text)

Max Activating Token Index: 690

Text #11

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1689. Min Act: -0.1700

Data Index: 2068206 (Open Web Text)

Max Activating Token Index: 362

Text #12

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1325. Min Act: -0.1700

Data Index: 6660355 (Open Web Text)

Max Activating Token Index: 113

Text #13

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1401. Min Act: -0.1700

Data Index: 975099 (Open Web Text)

Max Activating Token Index: 215

Text #14

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1366. Min Act: -0.1700

Data Index: 5982067 (Open Web Text)

Max Activating Token Index: 1011

Text #15

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1405. Min Act: -0.1700

Data Index: 4635959 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1773. Min Act: -0.1700

Data Index: 8822831 (Open Web Text)

Max Activating Token Index: 570

Text #17

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1188. Min Act: -0.1700

Data Index: 2733730 (Open Web Text)

Max Activating Token Index: 424

Text #18

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1008. Min Act: -0.1700

Data Index: 6637113 (Open Web Text)

Max Activating Token Index: 347

Text #19

Max Range: 4.5046. Min Range: -4.5046

Max Act: 4.1152. Min Act: -0.1700

Data Index: 3083922 (Open Web Text)

Max Activating Token Index: 308