Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 259 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1621. Min Act: -0.0000

Data Index: 558265 (The Pile)

Max Activating Token Index: 762

Text #1

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1791. Min Act: -0.0000

Data Index: 1519337 (The Pile)

Max Activating Token Index: 780

Text #2

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1777. Min Act: -0.0000

Data Index: 943122 (The Pile)

Max Activating Token Index: 875

Text #3

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1622. Min Act: -0.0000

Data Index: 1995545 (The Pile)

Max Activating Token Index: 933

Text #4

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1504. Min Act: -0.0000

Data Index: 177710 (The Pile)

Max Activating Token Index: 1016

Text #5

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1598. Min Act: -0.0000

Data Index: 446396 (The Pile)

Max Activating Token Index: 855

Text #6

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1487. Min Act: -0.0000

Data Index: 1679101 (The Pile)

Max Activating Token Index: 1007

Text #7

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1376. Min Act: -0.0000

Data Index: 25774 (The Pile)

Max Activating Token Index: 929

Text #8

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1293. Min Act: -0.0000

Data Index: 1419019 (The Pile)

Max Activating Token Index: 182

Text #9

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1222. Min Act: -0.0000

Data Index: 1544935 (The Pile)

Max Activating Token Index: 246

Text #10

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1292. Min Act: -0.0000

Data Index: 878736 (The Pile)

Max Activating Token Index: 485

Text #11

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1247. Min Act: -0.0000

Data Index: 1502915 (The Pile)

Max Activating Token Index: 460

Text #12

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1084. Min Act: -0.0000

Data Index: 1395073 (The Pile)

Max Activating Token Index: 977

Text #13

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1220. Min Act: -0.0000

Data Index: 483978 (The Pile)

Max Activating Token Index: 655

Text #14

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1184. Min Act: -0.0000

Data Index: 554813 (The Pile)

Max Activating Token Index: 502

Text #15

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1086. Min Act: -0.0000

Data Index: 426920 (The Pile)

Max Activating Token Index: 491

Text #16

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1256. Min Act: -0.0000

Data Index: 385518 (The Pile)

Max Activating Token Index: 505

Text #17

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1114. Min Act: -0.0000

Data Index: 503076 (The Pile)

Max Activating Token Index: 723

Text #18

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1234. Min Act: -0.0000

Data Index: 1968495 (The Pile)

Max Activating Token Index: 802

Text #19

Max Range: 0.1791. Min Range: -0.1791

Max Act: 0.1186. Min Act: -0.0000

Data Index: 1213466 (The Pile)

Max Activating Token Index: 1017